如果我有一个包含一列 Array[String] 的数据框:
scala> y.show
+---+----------+
|uid|event_comb|
+---+----------+
| c| [xx, zz]|
| b| [xx, xx]|
| b| [xx, yy]|
| b| [xx, zz]|
| b| [xx, yy]|
| b| [xx, zz]|
| b| [yy, zz]|
| a| [xx, yy]|
+---+----------+
如何拆分列
"event_comb"
分成两列(例如 "event1"
和 "event2"
)?
最佳答案
如果您的列类型是 list 或 Map 您可以使用 getItem 函数来获取值getItem(Object key)
An expression that gets an item at position ordinal out of an array, or gets a value by key key in a MapType.
val data = Seq(
("c", List("xx", "zz")),
("b", List("xx", "xx")),
("b", List("xx", "yy")),
("b", List("xx", "zz")),
("b", List("xx", "yy")),
("b", List("xx", "zz")),
("b", List("yy", "zz")),
("a", List("xx", "yy"))
).toDF("uid", "event_comb")
data.withColumn("event1", $"event_comb".getItem(0))
.withColumn("event2", $"event_comb".getItem(1))
.show(false)
输出:
+---+----------+------+------+
|uid|event_comb|event1|event2|
+---+----------+------+------+
|c |[xx, zz] |xx |zz |
|b |[xx, xx] |xx |xx |
|b |[xx, yy] |xx |yy |
|b |[xx, zz] |xx |zz |
|b |[xx, yy] |xx |yy |
|b |[xx, zz] |xx |zz |
|b |[yy, zz] |yy |zz |
|a |[xx, yy] |xx |yy |
+---+----------+------+------+
关于arrays - 将 Array[String] 的 Spark 列拆分为 String 列,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/48701289/