设置:
我有关于客户和他们最喜欢的 10 大电视节目的数据。到目前为止,我能够在 JavaRDD<Tuple2<String, Shows[]>>
中获取此数据.我能够打印它并检查它是否符合预期,确实如此。
目标:
现在,我需要将这些数据打印到一个文件中,格式如下:
Customer_1 Fav_TV_Show_1
Customer_1 Fav_TV_Show_2
Customer_1 Fav_TV_Show_3
Customer_1 Fav_TV_Show_4
Customer_2 Fav_TV_Show_1
Customer_2 Fav_TV_Show_2
Customer_2 Fav_TV_Show_3
Customer_2 Fav_TV_Show_4
Customer_3 Fav_TV_Show_1
Customer_3 Fav_TV_Show_2
Customer_3 Fav_TV_Show_3
Customer_3 Fav_TV_Show_4
问题:
我不知道该怎么做。到目前为止,我已经试过了:
// Need a flat pair back
JavaPairRDD<String, Shows> resultPairs = result.mapToPair(
new PairFunction<Tuple2<String,Shows[]>, String, Shows>() {
public Tuple2<String, Shows> call(Tuple2<String, Shows[]> t) {
// But this won't work as I have to return multiple <Customer - Show> pairs
}
});
}
非常感谢任何帮助。
最佳答案
好吧,你得到一个 JavaRDD<Tuple2<String, Shows[]>>
有点奇怪而不是 JavaPairRDD<String, Shows[]>
在键值对的情况下使用起来更舒服。尽管如此,您可以执行以下操作以使结果变平:
// convert your RDD into a PairRDD format
JavaPairRDD<String, Shows[]> pairs = result.mapToPair(new PairFunction<Tuple2<String,Shows[]>, String, Shows[]>() {
public Tuple2<String, Shows[]> call(Tuple2<String, Shows[]> t) throws Exception {
return t;
}
});
// now flatMap the values in order to split them with their respective keys
JavaPairRDD<String, Shows> output = pairs.flatMapValues(
new Function<Shows[], Iterable<Shows>>() {
public Iterable<Shows> call(Shows[] shows) throws Exception {
return Arrays.asList(shows);
}
});
// do something else with them
output.foreach(new VoidFunction<Tuple2<String, Shows>>() {
public void call(Tuple2<String, Shows> t) throws Exception {
System.out.println(t._1() + " " + t._2());
}
});
或者,您也可以获得 output
使用 flatMapToPair
的 RDD一步,手动组合 Shows
的数组进入 Iterable
如下:
JavaPairRDD<String, Shows> output = result.flatMapToPair(
new PairFlatMapFunction<Tuple2<String, Shows[]>, String, Shows>() {
public Iterable<Tuple2<String, Shows>> call(Tuple2<String, Shows[]> t) throws Exception {
ArrayList<Tuple2<String, Shows>> ret = new ArrayList<>();
for (Shows s : t._2())
ret.add(new Tuple2<>(t._1(), s));
return ret;
}
});
希望对您有所帮助。干杯!
关于java - 如何将 <String, Array[]> 打印成一对?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/29174607/