eclipse - Hadoop V2.7和Eclipse

标签 eclipse hadoop

我已经在Mac中设置了Hadoop v2.7,并且能够启动Hadoop守护程序。

我想使用eclipse编写MR程序,我需要一些帮助才能使Eclipse上的hadoop出现,我想知道要添加的jar文件和基本设置指南

以下是我的驱动程序类代码,我无法执行它

public class MyJobDriver extends Configured implements Tool {



    @Override
    public int run(String[] args) throws Exception {
        Configuration conf = getConf();
        JobConf job = new JobConf(conf, MyJobDriver.class);

        Path in = new Path(args[0]);
        Path out = new Path(args[1]);
        FileInputFormat.setInputPaths(job, in);
        FileOutputFormat.setOutputPath(job, out);

        job.setJobName("Patent");

        job.setMapperClass(InverseMapper.class);
        //Input Split consist two values separated by ","
        //K1 and V1 type is Text
        job.setInputFormat(KeyValueTextInputFormat.class);  
        job.set("key.value.separator.in.input.line",",");//Everything before the separator is the key and after is the value

        job.setOutputFormat(TextOutputFormat.class);//Key and value written as string and separated by tab(default)
        //when k1 and k2 are od same type and V1 and V2 are of same type
        //we can skip job.setMapOutputKeyClass() and job.setMapOutputValueClass()
        job.setOutputKeyClass(Text.class);
        job.setOutputValueClass(Text.class);

        //jobClient communicates with the JobTrackers to start job across clusters
        JobClient.runJob(job);
        return 0;
    }
        public static void main(String[] args) throws Exception {
            MyJobDriver driver = new MyJobDriver();
            System.out.println("Calling the run method");
             int exitCode = ToolRunner.run(driver, args);
             System.exit(exitCode);
    }

最佳答案

跟踪和检索必要的jar文件太麻烦了(有很多)。而是在eclipse中创建一个Maven项目并添加必要的依赖项,如此处https://hadoopi.wordpress.com/2013/05/25/setup-maven-project-for-hadoop-in-5mn/

关于eclipse - Hadoop V2.7和Eclipse,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/31225924/

相关文章:

java - Java eclipse 中调用者的条件断点

java - eclipse hell 。 . .读取项目描述文件(.project)失败

eclipse - 在 VM 中访问 Hadoop 后 Eclipse Europa 中的 DFS 位置

sql - HIVE Get Min 1列,并保留所有其他列

hadoop - 临时报告 Hadoop

hadoop - 使用Spark Sql查询hdfs

java - 用限定符第谷替换快照

java - 用java从其他项目导入类总是出错

hadoop - 如何正确控制YARN容器分配增量?

hadoop - 可以将 Hbase 用于 Hive 吗?