我试图在Java中的mapreduce 2.x中执行mapreduce单词计数示例。...我创建了jar,但是在执行它时显示了错误,例如在包中未找到WordMapper类,但我在包中声明了该错误。 ....请帮助我解决问题...
这是我的WordCount驱动程序代码:
package com.mapreduce2.x;
public class WordCount {
public static void main(String args[]) throws IOException, ClassNotFoundException, InterruptedException
{
Configuration conf=new Configuration();
org.apache.hadoop.mapreduce.Job job= new org.apache.hadoop.mapreduce.Job(conf, "Word_Count");
job.setMapperClass(WordMapper.class);
job.setReducerClass(WordReducer.class);
job.setMapOutputKeyClass(Text.class);
job.setMapOutputValueClass(IntWritable.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(IntWritable.class);
org.apache.hadoop.mapreduce.lib.input.FileInputFormat.setInputPaths(job, new Path(args[0]));
org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.setOutputPath(job, new Path(args[1]));
job.waitForCompletion(true);
}}
这是我的WordMapper类别:-
public class WordMapper extends Mapper<LongWritable, Text, Text,IntWritable>{
private final static IntWritable one=new IntWritable(1);
private Text word=new Text();
public void map(LongWritable key, Text value, org.apache.hadoop.mapreduce.Reducer.Context context) throws IOException, InterruptedException
{
String line=value.toString();
StringTokenizer tokenizer=new StringTokenizer(line);
while(tokenizer.hasMoreTokens())
{
word.set(tokenizer.nextToken());
context.write(word, one);
}
}}
WordReducer代码-
public class WordReducer extends Reducer<Text, IntWritable, Text, IntWritable>{
public void reduce(Text key, Iterator<IntWritable> values,Context context) throws IOException, InterruptedException
{
int sum =0;
while(values.hasNext())
{
sum= sum+values.next().get();
}
context.write(key, new IntWritable(sum));
}}
它正在执行时显示以下错误-
15/05/29 10:12:26 INFO mapreduce.Job: map 0% reduce 0%
15/05/29 10:12:33 INFO mapreduce.Job: Task Id : attempt_1432876892622_0005_m_000000_0, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class com.mapreduce2.x.WordMapper not found
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2076)
at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:186)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:742)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.ClassNotFoundException: Class com.mapreduce2.x.WordMapper not found
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1982)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2074)
... 8 more
最佳答案
在运行JAR文件时包括类名,或者可以在创建JAR文件时指定主类名。
如果运行时没有类名,则在运行JAR时指定类名。
使用命令
hadoop jar word.jar com.mapreduce2.x.WordMapper / input / output
这里的word.jar是JAR文件名。
要么
您还可以在创建jar文件时添加主类名称。
脚步:
文件->导出-> JAR->位置->然后单击下一步->要求选择主类->选择该类并单击确定
之后,您可以使用以下命令运行jar文件
hadoop jar word.jar /输入/输出
希望这能解决您的问题。
关于hadoop - 无法在mapreduce 2.x中运行wordcount示例,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/30527324/