hadoop - 在Windows中的Eclipse中调试hadoop Wordcount程序

标签 hadoop mapreduce

尝试在Eclipse的hadoop中运行Wordcount程序(Windows 7)。并仅在 eclipse 中传递这些参数

E:\hadoop\eclipse-hadoop-pro\workspace-hadoop\WordCountPro\input\word.txt
E:\hadoop\eclipse-hadoop-pro\workspace-hadoop\WordCountPro\output

我已经在项目中创建了输入文件,就像输入文件夹一样,并且在其中包含了word.txt文件

但是它正在经历以下的挑战
2015-04-08 15:30:09,947 WARN  [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2015-04-08 15:30:10,238 ERROR [main] util.Shell (Shell.java:getWinUtilsPath(373)) - Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable E:\hadoop\hadoop-HADOOP_HOME\hadoop-2.6.0\bin\bin\winutils.exe in the Hadoop binaries.
    at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:355)
    at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:370)
    at org.apache.hadoop.util.Shell.<clinit>(Shell.java:363)
    at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:79)
    at org.apache.hadoop.security.Groups.parseStaticMapping(Groups.java:104)
    at org.apache.hadoop.security.Groups.<init>(Groups.java:86)
    at org.apache.hadoop.security.Groups.<init>(Groups.java:66)
    at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:280)
    at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:271)
    at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:248)
    at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:763)
    at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:748)
    at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:621)
    at org.apache.hadoop.mapreduce.task.JobContextImpl.<init>(JobContextImpl.java:72)
    at org.apache.hadoop.mapreduce.Job.<init>(Job.java:144)
    at org.apache.hadoop.mapreduce.Job.getInstance(Job.java:187)
    at org.apache.hadoop.mapreduce.Job.getInstance(Job.java:206)
    at com.WordCount.main(WordCount.java:52)
2015-04-08 15:30:11,039 INFO  [main] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1049)) - session.id is deprecated. Instead, use dfs.metrics.session-id
2015-04-08 15:30:11,041 INFO  [main] jvm.JvmMetrics (JvmMetrics.java:init(76)) - Initializing JVM Metrics with processName=JobTracker, sessionId=
Exception in thread "main" org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory file:/E:/hadoop/eclipse-hadoop-pro/workspace-hadoop/WordCountPro/output already exists
    at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:146)
    at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:562)
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:432)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1293)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Unknown Source)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1293)
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
    at com.WordCount.main(WordCount.java:61)

最佳答案

我怀疑Hadoop是否正确安装。检查您的计算机是否所有守护程序都在运行。如果没有运行,请考虑重新检查或重新安装缺少的守护程序。

 ERROR [main] util.Shell (Shell.java:getWinUtilsPath(373)) - Failed to locate the winutils binary in the hadoop binary path java.io.IOException: Could not locate executable 

关于hadoop - 在Windows中的Eclipse中调试hadoop Wordcount程序,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/29512435/

相关文章:

MongoDB 的 mapReduce : partition keys to single reducers and affect key sorting

database - 我每天生成 2 GB 的 Web 服务器日志,如何过滤它们?

hadoop - 访问远程集群中的HDFS

hadoop - 目录为空时如何防止 Hadoop 作业失败?

java - 如何使用 mapreduce 批量更新满足查询的数据存储实体?

hadoop - Hadoop如何决定不。给定方案的 reducer 运行?

hadoop - 配置单元/DynamoDB 错误 "Could not find column mapping for column"

hadoop - 在Hadoop上安装SpatialHadoop框架

c# - 如何在 C# 中使用 Web HDFS REST API 附加文件?

java - 如何使用 Reactor 框架 2.x 执行多线程 map/reduce?