hadoop - 如何在 Hadoop 2.7.1 中获取作业计数器？

Question

我正在尝试使用 java 从特定作业中获取所有计数器... 我已经为 Hadoop 0.23.1 编写了工作代码:

    JobClient client = new JobClient(new JobConf(createConfiguration()));
    RunningJob job;
    system.out.print("Looking for job with title containing the string '" + jobName + "'");
    List<JobStatus> jobStatusList = Arrays.asList(client.getAllJobs());

    Collections.sort(jobStatusList, new Comparator<JobStatus>() {

        @Override
        public int compare(JobStatus o1, JobStatus o2) {
            return (o1.getStartTime() < o2.getStartTime()) ? 1 : (o1.getStartTime() > o2.getStartTime() ? -1 : 0);
        }
    });

    for (JobStatus jobStatus : jobStatusList) {
        job = client.getJob(jobStatus.getJobID());
        if (job.getJobName().contains(jobName)) {
            system.out.print(String.format("Job Name '%s' was found in job ID: %s...", jobName, jobStatus.getJobID().toString()));
            return job.getCounters();
        }
    }
    system.out.print("Did not find job that contains the string '" + jobName + "'. Counters are set to null.");

知道为什么它在 2.7.1 中不起作用吗？

Answer 1

成功使用 REST API:

http://<history server http address:port>/ws/v1/history/mapreduce/jobs/{jobid}/counters