hadoop - Pig 中的条件和

Question

我正在使用三元运算符有条件地在 SUM() 操作中包含值。这是我的做法。

GROUPED = GROUP ALL_MERGED BY (fld1, fld2, fld3);

REPORT_DATA = FOREACH GROUPED
              {     GENERATE group,
                    SUM(GROUPED.fld4 == 'S' ? GROUPED.fld5 : 0) AS sum1,
                    SUM(GROUPED.fld4 == 'S' ? GROUPED.fld5 : (GROUPED.fld5 * -1)) AS sum2;
               }

ALL_MERGED 的架构是

{ALL_MERGED: {fld1:chararray, fld2:chararray, fld3:chararray, fld4:chararray: fld5:int}}

当我执行它时，它会给我以下错误:

ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1000: Error during parsing. Invalid alias: SUM in {group: (fld1:chararray, fld2:chararray, fld3:chararray), ALL_MERGED: {fld1:chararray, fld2:chararray, fld3:chararray, fld4:chararray: fld5:int}}

我在这里做错了什么？

Answer 1

SUM 是一个将包作为输入的 UDF。您正在做的事情有很多问题，我怀疑它会帮助您查看有关 Pig 的良好引用。我推荐Programming Pig ，可在线免费获得。首先，GROUPED 有两个字段:一个名为 group 的元组和一个名为 ALL_MERGED 的包，这就是错误消息试图说明的内容你。 (我说“尝试”是因为 Pig 的错误消息通常非常含糊。)

此外，您不能像您希望的那样将表达式传递给 UDF。相反，您必须先GENERATE 这些字段，然后再传递它们。试试这个:

ALL_MERGED_2 =
    FOREACH ALL_MERGED
    GENERATE
        fld1 .. fld5,
        ((fld4 == 'S') ? fld5 : 0) AS sum_me1,
        ((fld4 == 'S') ? fld5 : fld5*-1) AS sum_me2;

GROUPED = GROUP ALL_MERGED_2 BY (fld1, fld2, fld3);
DATA =
    FOREACH GROUPED
    GENERATE
        group,
        SUM(ALL_MERGED_2.sum_me1) AS sum1,
        SUM(ALL_MERGED_2.sum_me2) AS sum2;