ubuntu - Hadoop 2.5.1 MapReduce卡住

标签 ubuntu hadoop mapreduce hdfs yarn

我在Ubuntu 14.04上并安装了Hadoop 2.5.1

我的jps命令为我提供了所有资源信息,因此我知道hadoop运行正常!

我尝试从here在分布式系统上运行MapReduce示例

但是在某点之后,工作冻结了

hduser@stratageeks-DL-E35LM1-R2:/usr/local/hadoop$ bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.5.1.jar grep input output 'dfs[a-z.]+'
14/09/24 00:13:02 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
14/09/24 00:13:14 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
14/09/24 00:13:20 WARN mapreduce.JobSubmitter: No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
14/09/24 00:13:21 INFO input.FileInputFormat: Total input paths to process : 30
14/09/24 00:13:25 INFO mapreduce.JobSubmitter: number of splits:30
14/09/24 00:13:30 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1411497776309_0001
14/09/24 00:13:33 INFO mapred.YARNRunner: Job jar is not present. Not adding any jar to the list of resources.
14/09/24 00:13:35 INFO impl.YarnClientImpl: Submitted application application_1411497776309_0001
14/09/24 00:13:35 INFO mapreduce.Job: The url to track the job: http://stratageeks-DL-E35LM1-R2:8088/proxy/application_1411497776309_0001/
14/09/24 00:13:35 INFO mapreduce.Job: Running job: job_1411497776309_0001

我甚至将yarn-site.xml中的内存使用量增加到4096,但仍然死机
<property>
<name>yarn.nodemanager.resource.memory-mb</name>
<value>4096</value>
<description>Physical memory, in MB, to be made available to running containers</description>
</property>

还有其他建议吗?

最佳答案

您是否检查了所有日志中的错误?还有userlogs文件夹下的特定于作业的日志(尤其是syslog)?
错误文件通常提供有关实际问题的良好线索。

关于ubuntu - Hadoop 2.5.1 MapReduce卡住,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/26002783/

相关文章:

ubuntu - DigitalOcean,Docker,Dokku : Installing Firefox inside a container

Hadoop MapReduce : Two values as key in Mapper-Reducer

java - Hadoop MR 作业 - java.lang.ClassNotFoundException : au. com.bytecode.opencsv.CSVParser

c - 尝试在 Ubuntu 中实现无名管道程序

python - 在ubuntu 18.04上安装python 2.7

regex - 使用脚本提取 Ubuntu 传感器命令

java - hadoop-aws 和 aws-java-sdk 版本与 Spark 2.3 兼容

hadoop - 基于 Cassandra 的数据分析和挖掘

maven - 凤凰构建失败。找不到标志。凤凰 4.8.2 Hbase-1.2 cdh 5.9.1

arrays - 遍历 Swift 数组并更改值