当我使用时:
context.write(key,value)
每行写一个“((键,值)”),但是我想更改它。
电流输出:
(key, value)
(key, value)
(key, value)
(key, value)
目标输出:
(key, value) (key, value) (key, value) (key, value)
键= NullWritable,值=像随机单词的文本
我该如何解决?
最佳答案
映射器输出:
(hi, 408)
(hi, 442)
(hi, 723)
(hi, 805)
最终/ reducer 输出:
(hi, 805) (hi, 723) (hi, 442) (hi, 408)
public class DataApp{
public static class DataMapper extends Mapper<Object, Text, NullWritable, Text> {
public void map(Object key, Text value, Context context) throws IOException, InterruptedException{
System.out.println("(hi, " + value.getLength() + ")");
context.write(NullWritable.get(), new Text("(hi, " + value.getLength() + ")"));
}
}
public static class DataReducer extends Reducer<NullWritable, Text, NullWritable, Text> {
public void reduce(NullWritable key, Iterable<Text> values, Context context)
throws IOException, InterruptedException {
String str="";
for(Text value: values){
str += value.toString() + " ";
}
context.write(NullWritable.get(), new Text(str));
}
}
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
Job job = Job.getInstance(conf, "stackoverflow-41476232");
job.setJarByClass(DataApp.class);
job.setMapperClass(DataMapper.class);
job.setReducerClass(DataReducer.class);
job.setMapOutputKeyClass(NullWritable.class);
job.setMapOutputValueClass(Text.class);
job.setOutputKeyClass(NullWritable.class);
job.setOutputValueClass(Text.class);
FileInputFormat.addInputPath(job, new Path(args[0]));
FileOutputFormat.setOutputPath(job, new Path(args[1]));
FileSystem fs = null;
Path dstFilePath = new Path(args[1]);
try {
fs = dstFilePath.getFileSystem(conf);
if (fs.exists(dstFilePath))
fs.delete(dstFilePath, true);
} catch (IOException e1) {
e1.printStackTrace();
}
job.waitForCompletion(true);
}
}
关于java - JAVA中的Hadoop MapReduce输出,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/41476232/