hadoop - Pig HCatLoader java.lang.String 无法转换为 org.apache.hadoop.hive.common.type.HiveVarchar

Question

我有下面的外部配置单元表，稍后我将使用它进行处理。

CREATE EXTERNAL TABLE hive_test_table(
  cmp_dte DATE,
  my_ind CHAR(1),
  typ_desc VARCHAR(40),
  def_typ_desc VARCHAR(60),
  tt_ind CHAR(1),
  nn_nm VARCHAR(80),
  xzs_desc VARCHAR(60),
  pqa_desc VARCHAR(40),
  aac_cnt INT,
  pqdd_cnt INT,
  nnc_cnt INT,
  ors_amt DECIMAL(18, 2),
  rddd_amt DECIMAL(18, 2),
  btdff_id BIGINT,
  fil_tss STRING
)
PARTITIONED BY( chnldd VARCHAR(40))
STORED AS ORC
LOCATION '/test/locationn'
;

要使用以下代码处理我想在 pig 中读取的数据，

existing_data = LOAD 'hive_test_table' USING org.apache.hive.hcatalog.pig.HCatLoader() ;

低于错误，

ERROR org.apache.pig.tools.grunt.Grunt - org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias existing_gold_paid_search_data. Backend error : Error converting read value to tuple
    at org.apache.pig.PigServer.openIterator(PigServer.java:872)
    at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:774)
    at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:372)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
    at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
    at org.apache.pig.Main.run(Main.java:541)
    at org.apache.pig.Main.main(Main.java:156)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error converting read value to tuple
    at org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:76)
    at org.apache.hive.hcatalog.pig.HCatLoader.getNext(HCatLoader.java:58)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:211)
    at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:533)
    at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
    at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to org.apache.hadoop.hive.common.type.HiveVarchar
    at org.apache.hive.hcatalog.pig.PigHCatUtil.extractPigObject(PigHCatUtil.java:413)
    at org.apache.hive.hcatalog.pig.PigHCatUtil.transformToTuple(PigHCatUtil.java:456)
    at org.apache.hive.hcatalog.pig.PigHCatUtil.transformToTuple(PigHCatUtil.java:374)
    at org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:64)

谁能帮忙，如何解决这个问题？

Answer 1

这是 Hive 版本 0.13 的已知问题

https://issues.apache.org/jira/browse/HIVE-11054