我有一个巨大的文本语料库(逐行),我想删除特殊字符但保留字符串的空间和结构。
hello? there A-Z-R_T(,**), world, welcome to python.
this **should? the next line#followed- by@ an#other %million^ %%like $this.
应该是
hello there A Z R T world welcome to python
this should be the next line followed by another million like this
您也可以将此模式与 regex
一起使用:
import re
a = '''hello? there A-Z-R_T(,**), world, welcome to python.
this **should? the next line#followed- by@ an#other %million^ %%like $this.'''
for k in a.split("\n"):
print(re.sub(r"[^a-zA-Z0-9]+", ' ', k))
# Or:
# final = " ".join(re.findall(r"[a-zA-Z0-9]+", k))
# print(final)
输出:
hello there A Z R T world welcome to python
this should the next line followed by an other million like this
编辑:
否则,您可以将最后几行存储到列表
中:
final = [re.sub(r"[^a-zA-Z0-9]+", ' ', k) for k in a.split("\n")]
print(final)
输出:
['hello there A Z R T world welcome to python ', 'this should the next line followed by an other million like this ']