java - getstrings方法hadoop中的空指针异常

Question

在 Driver 类 conf.getstrings() 方法中获取 Null 指针异常。这个驱动程序类是从我的自定义网站调用的。

下面是Driver类的详细信息

 @SuppressWarnings("unchecked")
public void doGet(HttpServletRequest request,
                 HttpServletResponse response)
         throws ServletException, IOException
{
       Configuration conf = new Configuration();       
       //conf.set("fs.default.name", "hdfs://localhost:54310");
       //conf.set("mapred.job.tracker", "localhost:54311");
       //conf.set("mapred.jar","/home/htcuser/Desktop/ResumeLatest.jar");
       Job job = new Job(conf, "ResumeSearchClass");        
       job.setJarByClass(HelloForm.class);      
       job.setJobName("ResumeParse");
       job.setInputFormatClass(FileInputFormat.class);
       FileInputFormat.addInputPath(job, new Path("hdfs://localhost:54310/usr/ResumeDirectory"));
       job.setMapperClass(ResumeMapper.class);
       job.setReducerClass(ResumeReducer.class);
       job.setMapOutputKeyClass(IntWritable.class);
       job.setSortComparatorClass(ReverseComparator.class);       
       job.setMapOutputValueClass(Text.class);
       job.setOutputKeyClass(IntWritable.class);
       job.setOutputValueClass(Text.class);
       job.setOutputFormatClass(FileOutPutFormat.class);
       FileOutputFormat.setOutputPath(job, new Path("hdfs://localhost:54310/usr/output" + System.currentTimeMillis()));
       long start = System.currentTimeMillis();
       var = job.waitForCompletion(true) ? 0 : 1;

下面两行代码获取NULL指针异常

String[] keytextarray=conf.getStrings("Keytext");   

for(int i=0;i<keytextarray.length;i++) //GETTING NULL POINTER EXCEPTION HERE IN keytextarray.length
    {
        //some code here
    }

       if(var==0)
       {        
        RequestDispatcher dispatcher = request.getRequestDispatcher("/Result.jsp");
        dispatcher.forward(request, response);
       long finish= System.currentTimeMillis();
       System.out.println("Time Taken "+(finish-start));
       }
}

我从上面的 Drives 类方法中删除了一些不需要的代码...

下面是 RecordWriter 类，我在 Write() 方法中使用 conf.setstrings() 来设置值

下面是 RecordWriter 类的详细信息

    public class RecordWrite extends org.apache.hadoop.mapreduce.RecordWriter<IntWritable, Text> {

    TaskAttemptContext context1;
    Configuration conf;

    public RecordWrite(DataOutputStream output, TaskAttemptContext context)
    {
        out = output;        
        conf  = context.getConfiguration();
        HelloForm.context1=context;        
        try {
            out.writeBytes("result:\n");
            out.writeBytes("Name:\t\t\t\tExperience\t\t\t\t\tPriority\tPriorityCount\n");            
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
    public RecordWrite() {
        // TODO Auto-generated constructor stub
    }
    @Override
    public void close(TaskAttemptContext context) throws IOException,
            InterruptedException
    {    
        out.close();
    }
    int z=0;
    @Override
    public void write(IntWritable value,Text key) throws IOException,
            InterruptedException
    {

        conf.setStrings("Keytext", key1string); //setting values here        
        conf.setStrings("valtext", valuestring);
        String[] keytext=key.toString().split(Pattern.quote("^"));
        //some code here

    }
}`

`我怀疑这个空指针异常发生是因为我在作业完成后调用了 conf.getstrings() 方法 (job.waitForCompletion(true))。请帮助解决这个问题。

如果上述代码不是将值从 recordwriter() 方法传递到驱动程序类的正确方法。请告诉我如何将值从 recordwriter() 传递到驱动程序类。

我尝试过将 RecordWriter() 中的值设置为自定义静态类的选项，如果我在集群中运行代码，则从 Driverclass 中的静态类访问该对象再次返回 Null 异常。

Answer 1

如果您在 Job 类中有 key1staring 和 valuestirng 的值，请尝试在作业类本身中设置它们，而不是 RecordWriter.write() 方法。