hadoop - Apache pig -错误 6007 : Unable to check name

Question

我正在尝试运行 Pig 教程 ( http://pig.apache.org/docs/r0.11.1/start.html#pig-scripts ) 中的基本脚本，如下所示:

/* myscript.pig
My script is simple.
It includes three Pig Latin statements.
*/

A = LOAD 'student' USING PigStorage() AS (name:chararray, age:int, gpa:float); -- loading data
B = FOREACH A GENERATE name;  -- transforming data
DUMP B;  -- retrieving results

输出:

2013-05-13 15:26:55,864 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 6007: Unable to check name hdfs://stage-hadoop101.cluster:8020/user/myusername
Details at logfile: /Volumes/Nimue/Environment/pig-0.11.1/pig_1368473213767.log
macbook-3:pig myusername$ cat /Volumes/Nimue/Environment/pig-0.11.1/pig_1368473213767.log

这导致:

Pig Stack Trace
---------------
ERROR 6007: Unable to check name hdfs://stage-hadoop101.cluster:8020/user/myusername

org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during parsing. Unable to check name hdfs://stage-hadoop101.cluster:8020/user/myusername
    at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1607)
    at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1546)
    at org.apache.pig.PigServer.registerQuery(PigServer.java:516)
    at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:991)
    at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:412)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:170)
    at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
    at org.apache.pig.Main.run(Main.java:604)
    at org.apache.pig.Main.main(Main.java:157)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: Failed to parse: Pig script failed to parse: 
<file test/myscript.pig, line 6, column 4> pig script failed to validate: org.apache.pig.backend.datastorage.DataStorageException: ERROR 6007: Unable to check name hdfs://stage-hadoop101.cluster:8020/user/myusername
    at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:191)
    at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1599)
    ... 14 more
Caused by: 
<file test/myscript.pig, line 6, column 4> pig script failed to validate: org.apache.pig.backend.datastorage.DataStorageException: ERROR 6007: Unable to check name hdfs://stage-hadoop101.cluster:8020/user/myusername
    at org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:835)
    at org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3236)
    at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1315)
    at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:799)
    at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:517)
    at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:392)
    at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:184)
    ... 15 more
Caused by: org.apache.pig.backend.datastorage.DataStorageException: ERROR 6007: Unable to check name hdfs://stage-hadoop101.cluster:8020/user/myusername
    at org.apache.pig.backend.hadoop.datastorage.HDataStorage.isContainer(HDataStorage.java:207)
    at org.apache.pig.backend.hadoop.datastorage.HDataStorage.asElement(HDataStorage.java:128)
    at org.apache.pig.backend.hadoop.datastorage.HDataStorage.asElement(HDataStorage.java:138)
    at org.apache.pig.parser.QueryParserUtils.getCurrentDir(QueryParserUtils.java:91)
    at org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:827)
    ... 21 more
Caused by: java.io.IOException: Failed on local exception: com.google.protobuf.InvalidProtocolBufferException: Message missing required fields: callId, status; Host Details : local host is: "macbook-3.local/192.168.2.2"; destination host is: "stage-hadoop101.cluster":8020; 
    at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:761)
    at org.apache.hadoop.ipc.Client.call(Client.java:1239)
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:202)
    at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:164)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:83)
    at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
    at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:630)
    at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1559)
    at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:811)
    at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1345)
    at org.apache.pig.backend.hadoop.datastorage.HDataStorage.isContainer(HDataStorage.java:200)
    ... 25 more
Caused by: com.google.protobuf.InvalidProtocolBufferException: Message missing required fields: callId, status
    at com.google.protobuf.UninitializedMessageException.asInvalidProtocolBufferException(UninitializedMessageException.java:81)
    at org.apache.hadoop.ipc.protobuf.RpcPayloadHeaderProtos$RpcResponseHeaderProto$Builder.buildParsed(RpcPayloadHeaderProtos.java:1094)
    at org.apache.hadoop.ipc.protobuf.RpcPayloadHeaderProtos$RpcResponseHeaderProto$Builder.access$1300(RpcPayloadHeaderProtos.java:1028)
    at org.apache.hadoop.ipc.protobuf.RpcPayloadHeaderProtos$RpcResponseHeaderProto.parseDelimitedFrom(RpcPayloadHeaderProtos.java:986)
    at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:946)
    at org.apache.hadoop.ipc.Client$Connection.run(Client.java:844)

搜索 Unable to check name 和 InvalidProtocolBufferException: Message missing required fields 没有找到任何结果。

Answer 1

如果您刚开始使用 pig，我建议您在 local mode first 中运行脚本.

 pig -x local myscript.pig

如果您想在 MapReduce 模式下运行它，请确保您已遵循 MapReduce mode instructions第一的。