环境详情:
我在 AWS(1 Namenode/1 2nd NN/3 datanodes - ubuntu 14.04)上安装了一个 hadoop 2.7.2(不是硬件，而是纯 Hadoop)多节点集群。
集群基于以下教程 (http://mfaizmzaki.com/2015/12/17/how-to-install-hadoop-2-7-1-multi-node-cluster-on-amazon-aws-ec2-instance-improved-part-1/) --> 这意味着第一次安装(主)被复制并调整
问题:
如果我使用 1 个 Datanode 配置集群(我特别排除了其他 2 个)，则 3 个数据节点可以单独正常工作。
一旦我添加另一个数据节点，数据节点启动首先会记录一个 fatal error (请参阅下文的日志文件摘录和 VERSION 文件的快照)并停止。数据节点启动第二个工作然后很好......
有什么想法推荐吗？
我在其他机器上克隆主设备的 AMI 时做错了什么吗？
谢谢各位!
日志文件
INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Unsuccessfully sent block report 0x1858458671b, containing 1 storage report(s), of which we sent 0. The reports had 0 total blocks and used 0 RPC(s). This took 5 msec to generate and 35 msecs for RPC and NN processing. Got back no commands.
WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool BP-1251070591-172.Y.Y.Y-1454167071207 (Datanode Uuid 54bc8b80-b84f-4893-8b96-36568acc5d4b) service to master/172.Y.Y.Y:9000 is shutting down org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.UnregisteredNodeException): Data node DatanodeRegistration(172.X.X.X:50010, datanodeUuid=54bc8b80-b84f-4893-8b96-36568acc5d4b, infoPort=50075, infoSecurePort=0, ipcPort=50020, storageInfo=lv=-56;cid=CID-8e09ff25-80fb-4834-878b-f23b3deb62d0;nsid=278157295;c=0) is attempting to report storage ID 54bc8b80-b84f-4893-8b96-36568acc5d4b. Node 172.Z.Z.Z:50010 is expected to serve this storage.
WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Ending block pool service for: Block pool BP-1251070591-172.31.34.94-1454167071207 (Datanode Uuid 54bc8b80-b84f-4893-8b96-36568acc5d4b) service to master/172.Y.Y.Y:9000
INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Removed Block pool BP-1251070591-172.Y.Y.Y-1454167071207 (Datanode Uuid 54bc8b80-b84f-4893-8b96-36568acc5d4b)
INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Removing block pool BP-1251070591-172.31.34.94-1454167071207
WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Exiting Datanode INFO org.apache.hadoop.util.ExitUtil: Exiting with status 0
INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down DataNode at HNDATA2/172.X.X.x ************************************************************/
最佳答案
您必须在 slaves 中添加三个数据节点的 IP 地址名称节点的文件。然后重启集群。这将解决问题
奴隶
<IPaddress of datanode1>
<IPaddress of datanode2>
<IPaddress of datanode3>
关于apache - Hadoop 272 - Datanodes 启动然后停止，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/35137894/