maven - Hadoop Maven 依赖错误

Question

我正在尝试使用 Maven 构建 Hadoop 作业。当我不使用 Maven 并直接将 Hadoop Jar 依赖项导入 eclipse 时，这项工作效果很好。

我还能够在没有 Hadoop 依赖项的情况下使用 Maven 构建一个简单的 jar(hello world 类型)。

但是当我添加 Hadoop 依赖项并运行 jar 时，我得到这个错误:

dataException in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
    at org.graphhadoop.CreateAdjacency.Adjacency(CreateAdjacency.java:74)
    at org.graphhadoop.Driver.main(Driver.java:12)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configuration
    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
    ... 2 more

pom 文件更改如下:

    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>3.8.1</version>
      <scope>test</scope>
        </dependency>
    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-core</artifactId>
        <version>0.20.203.0</version>
        </dependency>
  </dependencies>

<build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-jar-plugin</artifactId>
        <configuration>
          <archive>
            <manifest>
              <addClasspath>true</addClasspath>
              <mainClass>org.graphhadoop.Driver</mainClass>
            </manifest>
          </archive>
          <manifestEntries>
          <Class-Path>../target/classes</Class-Path>
        </manifestEntries>
        </configuration>
      </plugin>
    </plugins>
  </build>

我哪里错了？

编辑:

目前，我在本地运行 Jar - 不是在 Hadoop 上，而是在本地数据上。

java -jar jarname.jar

Answer 1

作为遇到相同问题的人的解决方案，以下是解决问题的方法:

需要将这些依赖添加到 pom.xml 中:

        <dependency>
        <groupId>org.codehaus.jackson</groupId>
        <artifactId>jackson-mapper-asl</artifactId>
        <version>1.5.2</version>
        </dependency>

        <dependency>
        <groupId>net.sf.json-lib</groupId>
        <artifactId>json-lib</artifactId>
        <version>2.3</version>
        <classifier>jdk15</classifier>
        </dependency>

        <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-core</artifactId>
        <version>0.20.203.0</version>
        </dependency>
  </dependencies>

和这些构建插件:

<build>
    <plugins>
    <plugin>
     <groupId>org.apache.maven.plugins</groupId>
         <artifactId>maven-assembly-plugin</artifactId>
        <executions>
        <execution>
         <goals>
           <goal>attached</goal>
         </goals>
         <phase>package</phase>
         <configuration>
           <descriptorRefs>
             <descriptorRef>jar-with-dependencies</descriptorRef>
          </descriptorRefs>
          <archive>
            <manifest>
              <addClasspath>true</addClasspath>
              <mainClass>org.lll..</mainClass>
            </manifest>
      <manifestEntries>
          <Class-Path>../target/classes</Class-Path>
            </manifestEntries>
          </archive>
        </configuration>
        </execution>
    </executions>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-jar-plugin</artifactId>
        <configuration>
          <archive>
            <manifest>
              <addClasspath>true</addClasspath>
          <mainClass>org...</mainClass>
            </manifest>
          </archive>
      <manifestEntries>
          <Class-Path>../target/classes</Class-Path>
        </manifestEntries>
        </configuration>
      </plugin>
    </plugins>  
</build>