apache-pig - 在 PIG 脚本中连接不同记录

Question

我正在尝试在 PIG 中编写一个脚本，我需要做的是获取一个数据集 - 其中包含用户 ID、日期、国家/地区代码和其他属性... 我想要的结果是按用户 ID 和日期进行分组，并且对于每个这样的组 - 将国家/地区代码连接到同一字段中...

例如:

user_id |  date    | country_code
1        2017-01-01     US
1        2017-01-01     UK
1        2017-01-02     FR
2        2017-01-02     RU
2        2017-01-03     DE
2        2017-01-03     AU

我想要的输出:

(1, 2017-01-01, "US,UK")
(1, 2017-01-02, FR)
(2, 2017-01-02, RU)
(2, 2017-01-03, "DE,AU")

Answer 1

一个措辞截然不同的问题实际上产生了 this answer作者:@Hari Shankar，由于问题似乎并不重复，我将直接在此处发布答案:

grouped = GROUP table BY userid;
   X = FOREACH grouped GENERATE group as userid, 
                                table.clickcount as clicksbag, 
                                table.pagenumber as pagenumberbag;

Now X will be:

{(155,{(2),(3),(1)},{(12),(133),(144)},
 (156,{(6),(7)},{(1),(5)}}

Now you need to use the builtin UDF BagToTuple:

output = FOREACH X GENERATE userid, 
                            BagToTuple(clickbag) as clickcounts, 
                            BagToTuple(pagenumberbag) as pagenumbers;

output should now contain what you want. You can merge the output step into the merge step as well:

    output = FOREACH grouped GENERATE group as userid, 
                     BagToTuple(table.clickcount) as clickcounts, 
                     BagToTuple(table.pagenumber) as pagenumbers;

1: http://pig.apache.org/docs/r0.11.1/api/org/apache/pig/builtin/BagToTuple.html