我在spark java中使用isin函数并传递id列表，需要按照传递列表的顺序检索id。但是使用isin函数后顺序发生了变化，它正在维护Dataset的顺序。
如何保留列表中的订单？
JavaSparkContext sc = new JavaSparkContext(new SparkConf().setAppName("SparkJdbcDs").setMaster("local[*]"));
SQLContext sqlContext = new SQLContext(sc);
SparkSession spark = SparkSession.builder().appName("JavaTokenizerExample").getOrCreate();
RowFactory.create("405-048011-62815", "CRC Industries"),
RowFactory.create("630-0746","Dixon value"),
RowFactory.create("4444-444","3M INdustries"),
RowFactory.create("4333-444","3M INdustries"),
RowFactory.create("4777-444","3M INdustries"),
RowFactory.create("4444-888","3M INdustries"),
RowFactory.create("4999-444","3M INdustries"),
RowFactory.create("5666-55","Dixon coupling valve"));
StructType schema = new StructType(new StructField[] {new StructField("label1", DataTypes.StringType, false,Metadata.empty()),
new StructField("sentence1", DataTypes.StringType, false,Metadata.empty()) });
Dataset<Row> sentenceDataFrame = spark.createDataFrame(data, schema);
List<String> listStrings = new ArrayList<String>();
listStrings.add("5666-55");
listStrings.add("630-0746");
listStrings.add("4777-444");
listStrings.add("4444-444");
Dataset<Row> matchFound1=sentenceDataFrame.filter(col("label1").isin(listStrings.stream().toArray(String[]::new)));
matchFound1.show();
当前输出:
+--------+--------------------+
| label1| sentence1|
+--------+--------------------+
|630-0746| Dixon value|
|4444-444| 3M INdustries|
|4777-444| 3M INdustries|
| 5666-55|Dixon coupling valve|
+--------+--------------------+
预期输出:
+--------+--------------------+
| label1| sentence1|
+--------+--------------------+
| 5666-55|Dixon coupling valve|
|630-0746| Dixon value|
|4777-444| 3M INdustries|
|4444-444| 3M INdustries|
+--------+--------------------+
最佳答案
我建议您创建一个dataframe而不是list并与sentenceDataFrame连接，您也将保留顺序。这比创建列表和过滤更有效。
关于java - 在spark java中使用isin函数时保留列表的顺序，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/44075059/