我正在尝试使用最新的 Giraph 代码和 Hadoop 2.5.2 运行 SimpleShortestPathsComputation 示例。我的命令行如下所示:
hadoop jar /home/prhodes/giraph/giraph-examples/target/giraph-examples-1.2.0-SNAPSHOT-for-hadoop-2.5.2-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/prhodes/input/tiny_graph.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/prhodes/giraph_output/shortestpaths -w 4
作业似乎开始正常。但随后它开始输出这些类型的消息,并且这种情况(看似)一直持续到您按 ctrl+c 为止。
15/03/11 02:54:31 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed:
305.43 secs 15/03/11 02:54:31 INFO yarn.GiraphYarnClient: appattempt_1426041786848_0002_000001, State: ACCEPTED, Containers
used: 1 15/03/11 02:54:35 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed:
309.44 secs 15/03/11 02:54:35 INFO yarn.GiraphYarnClient: appattempt_1426041786848_0002_000001, State: ACCEPTED, Containers
used: 1 15/03/11 02:54:39 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed:
313.45 secs 15/03/11 02:54:39 INFO yarn.GiraphYarnClient: appattempt_1426041786848_0002_000001, State: ACCEPTED, Containers
used: 1 15/03/11 02:54:43 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed:
317.45 secs 15/03/11 02:54:43 INFO yarn.GiraphYarnClient: appattempt_1426041786848_0002_000001, State: ACCEPTED, Containers
used: 1 ^C15/03/11 02:54:47 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed:
321.46 secs 15/03/11 02:54:47 INFO yarn.GiraphYarnClient: appattempt_1426041786848_0002_000001, State: ACCEPTED, Containers
used: 1
知道这里发生了什么吗?
最佳答案
事实证明这是一个环境问题。我试图在我的 AWS 安全组中挑选端口加入白名单,但错过了 YARN 使用一些端口在集群上进行内部通信。我没有尝试对整个列表进行排序,而是添加了一个安全组规则,以便从 10.0.0.0/24 接受“所有流量”。这解决了这个问题。
关于java - Giraph 的工作永无止境,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/28977896/