我尝试在 Windows 7 计算机上运行 PigUnit 测试,然后再在 Ubuntu 集群上运行实际的 Pig 脚本,我开始认为我对“withouthadoop”的理解不正确。 p>
Do I need to install Hadoop to locally run a PigUnit test on a Windows 7 machine?
我安装了:
- eclipse 朱诺和 Ant
- cygwin
我设置:
JAVA_HOME=C:\Program Files\Java\jdk1.6.0_39
PIG_HOME=C:\Users\john.doe\Java\eclipse\pig
PIG_CLASSPATH=%PIG_HOME%\bin
我使用 eclipse 的 Ant 构建器 jar-all
和 pigunit-jar
创建:
pig.jar
pig-withouthadoop.jar
pigunit.jar
当我在 cygwin 中输入 pig -x local
时,我得到:
$./pig -x local
cygpath: can't convert empty path
Exception in thread "main" java.io.IOException: Error opening job jar: /usr/lib/pig/pig-withouthadoop.jar
at org.apache.hadoop.util.RunJar.main(RunJar.java:135)
Caused by: java.io.FileNotFoundException: \usr\lib\pig\pig-withouthadoop.jar (the systen cannot find the given path)
at java.util.zip.ZipFile.open(Native Method)
at java.util.zip.ZipFile.<init>(ZipFile.java:127)
at java.util.jar.JarFile.<init>(JarFile.java:136)
at java.util.jar.JarFile.<init>(JarFile.java:73)
at org.apache.hadoop.util.RunJar.main(RunJar.java:133)
当我尝试从 http://pig.apache.org/docs/r0.10.0/test.html#pigunit 运行测试时从 Eclipse 中使用选项“作为 JUnit 运行”,我得到:
java.io.IOException
at org.apache.pig.pigunit.pig.PigServer.registerScript(PigServer.java:62)
at org.apache.pig.pigunit.PigTest.registerScript(PigTest.java:171)
at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:267)
at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:262)
at da.utils.pigunit.PigUnitExample.testTop2Queries(PigUnitExample.java:72)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
我开始认为我错过了一些在 Windows 上运行 Pig 的关键基本信息,我还必须说,我不是 Windows 7 和 cygwin 的经验丰富的用户,我来自 Unix 世界。
最佳答案
别反抗。从 Web 平台安装程序在 Windows 上安装 Hadoop HDInsight 服务器: http://www.microsoft.com/web/downloads/platform.aspx
它不会花费很长时间或占用那么多空间,整个 shebang 就已经为您设置好了并运行。我无法让 Pig 脚本接受参数,也没有 HBase,但你可以使用 HDFS、Pig、Hive。你甚至可以让整个本地集群运行起来,只要遵循::http://social.msdn.microsoft.com/Forums/en-US/hdinsight/thread/885efc22-fb67-4df8-8648-4ff38098dac6/
关于windows - Windows 7 上没有 Hadoop 的 Pig,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/15992472/