apache-spark - 从 RDD 中提取特定的主题值

Question

我正在尝试从以下 kafka 主题中阅读 spark :

Map<TopicAndPartition, Long> map = new HashMap<>();
        map.put(new TopicAndPartition("A", 0), 1L);
        map.put(new TopicAndPartition("B", 0), 1L);

        JavaInputDStream<Map.Entry> topicMessages = KafkaUtils.createDirectStream(
                                                                            jssc,
                                                                            String.class,
                                                                            String.class,
                                                                            StringDecoder.class,
                                                                            StringDecoder.class,
                                                                            Map.Entry.class,
                                                                            kafkaParams,
                                                                            map,
                                                                            messageAndMetadata -> 
                                                                                new AbstractMap.SimpleEntry<>(messageAndMetadata.topic(),
                                                                                                              messageAndMetadata.message())
                                                                          );

现在 topicMessage 具有如下所示的键和值格式的所有值:

A="04/15/2015","18:44:28"
A="04/15/2015","18:44:28"
A="04/15/2015","18:44:28"
B="04/15/2016","18:44:28"
B="04/15/2014","18:44:28"  

如何提取特定主题的值(value)。
对于名为 B

的主题，如下所示

"04/15/2016","18:44:28"
"04/15/2014","18:44:28" 

Answer 1

如果您想要给定主题的行，您只需要做:

JavaPairDStream<String> rowsFromTopicB = topicMessages.filter( entry -> entry.getKey().toString().equals("B")).map(entry -> entry.getValue().toString())