在 Hadoop 中，我将结果整理到主节点上，然后准备写入 S3，出现以下错误:
18/08/19 20:10:13 WARN DataStreamer: Exception for BP-2033025931-192.168.3.27-1534682170082:blk_1073741835_1011 java.io.EOFException: Unexpected EOF while trying to read response from server at org.apache.hadoop.hdfs.protocolPB.PBHelperClient.vintPrefixed(PBHelperClient.java:402) at org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:213) at org.apache.hadoop.hdfs.DataStreamer$ResponseProcessor.run(DataStreamer.java:1073)
18/08/19 20:10:13 WARN DataStreamer: Error Recovery for BP-2033025931-192.168.3.27-1534682170082:blk_1073741835_1011 in pipeline [DatanodeInfoWithStorage[192.168.3.141:50010,DS-fe8dd87d-2b16-4746-b5ee-c65a836cebb4,DISK], DatanodeInfoWithStorage[192.168.3.183:50010,DS-54ceb3d7-fdc3-4a8e-8df1-ab9c53f25628,DISK], DatanodeInfoWithStorage[192.168.3.106:50010,DS-4640296a-8a2a-4223-9759-f592aca85e77,DISK]]: datanode 0(DatanodeInfoWithStorage[192.168.3.141:50010,DS-fe8dd87d-2b16-4746-b5ee-c65a836cebb4,DISK]) is bad.
然后出现第二个错误:
com.amazonaws.services.s3.model.AmazonS3Exception: Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed. (Service: Amazon S3; Status Code: 400; Error Code: RequestTimeout; Request ID: BF10123C46AC4; S3 Extended Request ID: SXF+/rJOtW8714PGg0GEz9bgJuKuf/2BNOskx0JdTa02g1a/3igd3vfuaO7zI=)
这两个requestID并不完整，因为我不知道他们能为他们做些什么。
最佳答案
这次再运行就没有这个错误了，我猜这是一个偶然事件，然后我添加了如下代码:
ClientConfiguration s3ClientConfig = new ClientConfiguration().withMaxErrorRetry(10)
.withConnectionTimeout(3 * 60 * 1000)
.withSocketTimeout(3 * 60 * 1000);
我希望它能在未来发挥作用。
关于amazon-web-services - 在亚马逊。我将结果整理到主节点上，然后准备写入 S3，出现以下错误 :，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/51923724/