当我编译 hadoop 命令时，它最终出现以下错误， “java.io.IOException:映射中的键类型不匹配:预期为 org.apache.hadoop.io.Text，收到了 org.apache.hadoop.io.LongWritable”
我将数据类型从 Text 更改为 LongWritable，在这种情况下，我会遇到其他数据类型不匹配的情况。
主类:
public class CalculateMaximum {
public static void main(String [] args) throws IllegalArgumentException, IOException, ClassNotFoundException, InterruptedException{
Configuration config = new Configuration();
Job job = new Job(config);
job.setJarByClass(CalculateMaximum.class);
job.setMapperClass(CalculateMapper.class);
job.setNumReduceTasks(1);
job.setReducerClass(CalReducer.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(IntWritable.class);
FileInputFormat.addInputPath(job, new Path(args[0]));
FileOutputFormat.setOutputPath(job, new Path(args[1]));
FileSystem fs = FileSystem.get(config);
fs.delete(new Path(args[1]));
job.waitForCompletion(true);
}
}
映射器类:
public class CalculateMapper extends Mapper<LongWritable,Text,Text,IntWritable> {
public void cal(LongWritable key,Text values,Context context) throws IOException, InterruptedException{
String row = values.toString();
String []r1 = row.split(" ");
//Integer year = Integer.parseInt(row[0]);
Text yr = new Text(r1[0]);
Integer temp = Integer.parseInt(r1[1]);
IntWritable tp = new IntWritable(temp);
context.write(yr, tp);
//context.write(yr, tp);
}
}
reducer 类别:
public class CalReducer extends Reducer<Text,Iterable<IntWritable>,Text,IntWritable> {
public void cal(Text key,Iterable<IntWritable> values,Context context) throws IOException, InterruptedException{
//Iterable<IntWritable> tmps = values;
//int temp = tmps.get();
int max = 0;
for(IntWritable temp : values){
if(temp.get() > max){
max= temp.get();
}
context.write(key, new IntWritable(max));
}
}
}
我的输入数据如下:
1900 39
1900 14
1900 5
1900 11
1901 32
1901 40
1901 29
1901 48
预期输出:
1900 39
1901 48
最佳答案
我相信 key 和 value 都是 int 类型。您可以尝试使用 IntWritable 作为 key 吗？
关于java - 我的 MapReduce 程序出现错误，我想收集多年来的最高温度，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/57193288/