我也尝试在本地和 hdfs 模式下对 pig 中的别名运行 illustrate 命令。 但是我遇到了以下错误。
2014-08-27 19:18:06,703 [main] ERROR org.apache.pig.pen.ExampleGenerator - Error reading data. Internal error creating job configuration. java.lang.RuntimeException: Internal error creating job configuration. at org.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:160) at org.apache.pig.PigServer.getExamples(PigServer.java:1182) at org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:739) at org.apache.pig.tools.pigscript.parser.PigScriptParser.Illustrate(PigScriptParser.java:626) at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:323) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:170) at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69) at org.apache.pig.Main.run(Main.java:538) at org.apache.pig.Main.main(Main.java:157) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:208) 2014-08-27 19:18:06,707 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2997: Encountered IOException. Exception Details at logfile: /opt/pig_1409147241095.log
我正在使用此示例运行 Illustrate 命令。
假设输入文件是包含以下数据的“visits.txt”:
Amy cnn.com 20070218 Fred harvard.edu 20071204 Amy
bbc.com 20071205 Fred stanford.edu 20071206
grunt session 可能看起来像这样(注意加载数据时模式的使用。ExampleGenerator 需要您提供别名):
grunt> visits = load 'visits.txt' as (user, url, timestamp);
grunt> recent_visits = filter visits by timestamp >= '20071201';
grunt> user_visits = group recent_visits by user;
grunt> num_user_visits = foreach user_visits generate group, COUNT(recent_visits);
grunt> illustrate num_user_visits
最佳答案
在 /etc/pig/conf/pig.properties 上设置 pig.enable.plan.serialization=false
关于hadoop - 如何在 pig 中运行说明命令？，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/25529069/