java - 如何保持 DataFrame 中的键值顺序与 JSON 相同？

Question

示例 JSON 数据:

{"name": "dev","salary": 100,"occupation": "engg","address": "noida"}
{"name": "karthik","salary": 200,"occupation": "engg","address": "blore"}

Spark Java 代码:

DataFrame df = sqlContext.read().json(jsonPath);
df.printSchema();
df.show(false);

输出:

root
 |-- address: string (nullable = true)
 |-- name: string (nullable = true)
 |-- occupation: string (nullable = true)
 |-- salary: long (nullable = true)


+-------+-------+----------+------+
|address|name   |occupation|salary|
+-------+-------+----------+------+
|noida  |dev    |engg      |10000 |
|blore  |karthik|engg      |20000 |
+-------+-------+----------+------+

列按字母顺序排列。 有什么方法可以维持自然秩序吗？

Answer 1

您可以在读取json 时提供schema，它将保持顺序。

StructType schema = DataTypes.createStructType(new StructField[] { 
    DataTypes.createStructField("name", DataTypes.StringType, true),
    DataTypes.createStructField("salary", DataTypes.IntegerType, true),
    DataTypes.createStructField("occupation", DataTypes.StringType, true),
    DataTypes.createStructField("address", DataTypes.StringType, true)});

DataFrame df = sqlContext.read().schema(schema).json(jsonPath);
df.printSchema();
df.show(false);