python - 更改数据类型但返回 Dataframe 的空值

Question

我是 Pyspark 3.0 的新手，我有一项作业，需要将字符串 (geolocation) 更改为元组数字数据类型 (geolocation1)。

这是我的代码:

from pyspark.sql.functions import *
from pyspark.sql.types import *

df = df2.withColumn('geolocation1', col('geolocation').cast('double'))

输出:

|地理定位             |地理位置1 |
| ------------------------------------------------------| ----------------- |
| (-37.80899950, 140.96004459) |空                |
| (-37.80899952, 140.96004451) |空                |

我在这里做错了什么？

Answer 1

如果你有这样的字符串，你可以删除括号并用逗号分隔，然后转换为 array<double> :

import pyspark.sql.functions as F

df = df2.withColumn(
    'geolocation1', 
    F.split(
        F.regexp_replace('geolocation', '[\( \)]', ''), 
        ','
    ).cast('array<double>')
)

df.show(truncate=False)
+----------------------------+---------------------------+
|geolocation                 |geolocation1               |
+----------------------------+---------------------------+
|(-37.80899950, 140.96004459)|[-37.8089995, 140.96004459]|
+----------------------------+---------------------------+

df.printSchema()
root
 |-- geolocation: string (nullable = false)
 |-- geolocation1: array (nullable = false)
 |    |-- element: double (containsNull = true)