python - 用 replace_regex 替换 pyspark 中的括号

Question

+---+------------+
|  A|           B|
+---+------------+
| x1|        [s1]|
| x2|   [s2 (A2)]|
| x3|   [s3 (A3)]|
| x4|   [s4 (A4)]|
| x5|   [s5 (A5)]|
| x6|   [s6 (A6)]|
+---+------------+

想要的结果:

+---+------------+-------+
|A  |B           |value  |
+---+------------+-------+
|x1 |[s1]        |[s1]   |
|x2 |[s2 (A2)]   |[s2]   |
|x3 |[s3 (A3)]   |[s3]   |
|x4 |[s4 (A4)]   |[s4]   |
|x5 |[s5 (A5)]   |[s5]   |
|x6 |[s6 (A6)]   |[s6]   |
+---+------------+-------+

当我应用下面的每个代码时，它们之前的括号和空格没有被替换:

from pyspark.sql.functions import expr
df.withColumn("C",
               expr('''transform(B, x-> regexp_replace(x, ' \\(A.\\)', ''))''')).show(truncate=False)

或者

df.withColumn("C",
               expr('''transform(B, x-> regexp_replace(x, ' \(A.\)', ''))''')).show(truncate=False)

得到的结果:

+---+------------+------------+
|A  |B           |value       |
+---+------------+------------+
|x1 |[s1]        |[s1]        |
|x2 |[s2 (A2)]   |[s2 ()]     |
|x3 |[s3 (A3)]   |[s3 ()]     |
|x4 |[s4 (A4)]   |[s4 ()]     |
|x5 |[s5 (A5)]   |[s5 ()]     |
|x6 |[s6 (A6)]   |[s6 ()]     |
+---+------------+------------+

Answer 1

您可以创建一个 UDF，从数组中删除与正则表达式 r"\(.*\)" 匹配的所有元素。 .如有必要，您可以更改正则表达式以匹配 r"\(A.\)"如果需要的话。

import re
replaced = F.udf(lambda arr: [s for s in arr if not re.compile(r"\(.*\)").match(s)], \
                 T.ArrayType(T.StringType()))
df.withColumn("value", replaced("B")).show()