java - NoClassDefFoundError与gradle，giraph和hadoop

Question

因此，我一直到处逛逛，但找不到很好的答案，这使我发疯，所以我想在这里问一下，希望能得到帮助。我正在尝试使用gradle在Giraph项目中进行自动化测试。我是gradle的初学者。刚开始，我将SimpleShortestPathComputation类的测试代码复制到了我的项目中，以确保可以启动并运行测试。但是，当我执行gradle test时，出现以下错误:

$ gradle test --info
<skipping some output here...>

Successfully started process 'Gradle Test Executor 1'
Gradle Test Executor 1 started executing tests.
WCCTest > testToyData FAILED
    java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory
        at org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:142)
        at WCCTest.testToyData(WCCTest.java:180)

    Caused by:
    java.lang.ClassNotFoundException: org.apache.commons.logging.LogFactory
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
        ... 2 more

Gradle Test Executor 1 finished executing tests.
WCCTest > testOnShorterPathFound FAILED
    java.lang.NoClassDefFoundError: org.apache.hadoop.conf.Configuration
        at sun.reflect.GeneratedSerializationConstructorAccessor33.newInstance(Unknown Source)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
        at     org.objenesis.instantiator.sun.SunReflectionFactoryInstantiator.newInstance(SunReflectionFactoryInstantiator.java:40)
        at org.objenesis.ObjenesisBase.newInstance(ObjenesisBase.java:59)
        at org.mockito.internal.creation.jmock.ClassImposterizer.createProxy(ClassImposterizer.java:128)
        at org.mockito.internal.creation.jmock.ClassImposterizer.imposterise(ClassImposterizer.java:63)
        at org.mockito.internal.creation.jmock.ClassImposterizer.imposterise(ClassImposterizer.java:56)
        at org.mockito.internal.creation.CglibMockMaker.createMock(CglibMockMaker.java:23)
        at org.mockito.internal.util.MockUtil.createMock(MockUtil.java:26)
        at org.mockito.internal.MockitoCore.mock(MockitoCore.java:51)
        at org.mockito.Mockito.mock(Mockito.java:1243)
        at org.mockito.Mockito.mock(Mockito.java:1120)
        at org.apache.giraph.utils.MockUtils$MockedEnvironment.<init>(MockUtils.java:68)
        at org.apache.giraph.utils.MockUtils.prepareVertexAndComputation(MockUtils.java:132)
        at WCCTest.testOnShorterPathFound(WCCTest.java:64)

WCCTest > testToyDataJson FAILED
    java.lang.NoClassDefFoundError: Could not initialize class org.apache.giraph.conf.GiraphConfiguration
        at WCCTest.testToyDataJson(WCCTest.java:127)

WCCTest > testOnNoShorterPathFound FAILED
    java.lang.NoClassDefFoundError: org.apache.hadoop.conf.Configuration
        at sun.reflect.GeneratedSerializationConstructorAccessor33.newInstance(Unknown Source)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
        at org.objenesis.instantiator.sun.SunReflectionFactoryInstantiator.newInstance(SunReflectionFactoryInstantiator.java:40)
        at org.objenesis.ObjenesisBase.newInstance(ObjenesisBase.java:59)
        at org.mockito.internal.creation.jmock.ClassImposterizer.createProxy(ClassImposterizer.java:128)
        at org.mockito.internal.creation.jmock.ClassImposterizer.imposterise(ClassImposterizer.java:63)
        at org.mockito.internal.creation.jmock.ClassImposterizer.imposterise(ClassImposterizer.java:56)
        at org.mockito.internal.creation.CglibMockMaker.createMock(CglibMockMaker.java:23)
        at org.mockito.internal.util.MockUtil.createMock(MockUtil.java:26)
        at org.mockito.internal.MockitoCore.mock(MockitoCore.java:51)
        at org.mockito.Mockito.mock(Mockito.java:1243)
        at org.mockito.Mockito.mock(Mockito.java:1120)
        at org.apache.giraph.utils.MockUtils$MockedEnvironment.<init>(MockUtils.java:68)
        at org.apache.giraph.utils.MockUtils.prepareVertexAndComputation(MockUtils.java:132)
        at WCCTest.testOnNoShorterPathFound(WCCTest.java:95)

4 tests completed, 4 failed
<more output...>
:test FAILED
:test (Thread[main,5,main]) completed. Took 1.663 secs.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':test'.
> There were failing tests. See the report at: file:///...build/reports/tests/index.html

BUILD FAILED

我正在使用完全标准的项目目录结构，这是我的build.gradle文件:

apply plugin: 'java'

repositories {
    mavenCentral()
}

dependencies {
    compile files('$GIRAPH_HOME/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar')
    compile files('$GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar')
    compile files('$HADOOP_HOME/hadoop-core-0.20.203.0.jar')

    testCompile group: 'junit', name: 'junit', version: '4.+'
    testCompile group: 'org.mockito', name: 'mockito-all', version: '1.9.5'
}

它可以毫无问题地进行编译，作为依赖，我包含的jar文件包括它说NoClassDefFoundError的类(根据jar tf)。有什么想法我做错了吗？提前致谢。

Answer 1

尝试将以下内容添加到您的build.gradle中

    println("HADOOP_HOME=$HADOOP_HOME")
    compile files("$HADOOP_HOME/hadoop-core-0.20.203.0.jar")
    println("System.env.HADOOP_HOME=$System.env.HADOOP_HOME")
    compile files("$System.env.HADOOP_HOME/hadoop-core-0.20.203.0.jar")