java - 尝试使用 JAVA 启动 Pig 脚本时出错

Question

我正在尝试从 JAVA 启动 pig 脚本。这是我的代码:

import java.io.IOException;
import java.util.Properties;
import org.apache.pig.ExecType;
import org.apache.pig.PigServer;
import org.apache.pig.backend.executionengine.ExecException;


public class pigCV {


public static void main(String args[]){

    PigServer pigServer;
    try {

        Properties props = new Properties();
        props.setProperty("fs.default.name", "hdfs://hdfs://localhost:8022");
        props.setProperty("mapred.job.tracker", "localhost:8021");

        pigServer = new PigServer(ExecType.MAPREDUCE, props);

        pigServer.registerScript("Desktop/text_v3.pig");

    } 
    catch (ExecException e) {   e.printStackTrace(); } 
    catch (IOException e) { e.printStackTrace(); }

}


}

但是会抛出一些异常:

2013-05-23 01:34:54,666 ERROR [main] conf.Configuration(1151): Failed to set setXIncludeAware(true) for parser org.apache.xerces.jaxp.DocumentBuilderFactoryImpl@1787038:java.lang.UnsupportedOperationException: This parser does not support specification "null" version "null" java.lang.UnsupportedOperationException: This parser does not support specification "null" version "null" at javax.xml.parsers.DocumentBuilderFactory.setXIncludeAware(DocumentBuilderFactory.java:590) at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1149) at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1125) at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1064) at org.apache.hadoop.conf.Configuration.get(Configuration.java:424) at org.apache.hadoop.mapred.JobConf.checkAndWarnDeprecation(JobConf.java:1709) at org.apache.hadoop.mapred.JobConf.(JobConf.java:164) at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:169) at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:137) at org.apache.pig.impl.PigContext.connect(PigContext.java:200) at org.apache.pig.PigServer.(PigServer.java:169) at org.apache.pig.PigServer.(PigServer.java:158) at org.apache.pig.PigServer.(PigServer.java:154) at pigCV.main(pigCV.java:21)

你有什么想法可以帮助我吗？

谢谢。

Answer 1

您的类路径中可能有一个(旧的)Xerces 实现。 尝试设置

-Djavax.xml.parsers.DocumentBuilderFactory=
   com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl

(在 Eclipse 中:VM 参数)或在代码中:

System.setProperty("javax.xml.parsers.DocumentBuilderFactory",
  "com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl");

备注:
fs.default.name 可能有错别字:不应该是 hdfs://localhost:8022 而不是 hdfs://hdfs://本地主机:8022 ?