我将处理数千个 XML 文件,它们的格式相似,但父名称和父数量不同。 通过书籍、谷歌、教程和尝试代码,我已经能够提取所有这些数据。参见,例如:Parsing xml to pandas data frame throws memory error和 Dynamic search through xml attributes using lxml and xpath in python
但是,我意识到我提取的数据很糟糕,每个父项都重复了一个子项“Time”。
这就是我想要得到的。
Time blah abc
1200 100 2
1300 30 4
1400 70 2
这是我知道如何得到的。但我目前的方法很笨拙(我将在示例 XML 下方展示)
child Time grandchild
0 blah 1200 100
1 blah 1300 30
...
n-2 abc 1200 2
n-1 abc 1300 4
n abc 1400 2
示例 XML 格式
<outer>
<inner>
<parent name = "blah" id = "1">
<child Time = "1200">
<grandchild>100</grandchild>
</child>
<child Time = "1300">
<grandchild>30</grandchild>
</child>
<child Time = "1400">
<grandchild>70</grandchild>
</child>
</parent>
<parent name = "abc" id = "2">
<child Time = "1200">
<grandchild>2</grandchild>
</child>
<child Time = "1300">
<grandchild>4</grandchild>
</child>
<child Time = "1400">
<grandchild>2</grandchild>
</child>
</parent>
<parent name = "1234" id = "7734">
<other> 12 </other>
</parent>
</inner>
</outer>
这里是我如何得到我的输出:
from lxml import etree, objectify
from pandas import *
dTime=[]
dparent = []
dgrandchild=[]
for df in root.xpath('/*/*/*/parent/child'):
dparent.append(df.getparent().attrib['name'])
## Iterate over attributes of time for specific parent
for attrib in df.attrib:
dTime.append(df.attrib[attrib])
## grandchild is a child of time, and iterate
subfields = df.getchildren()
for subfield in subfields:
dgrandchild.append(subfield.text)
df=DataFrame({'Parent': dparent,'Time':dTime,'grandchild':dgrandchld})
我可以只接受这个输出并重新塑造它,但这似乎效率低下并且是一种非常笨拙的方法。
我想我需要一些味道:
#this does not work
data = []
for elem in root.xpath('/*/*/*/parent/child'):
elem_data = {}
for attrib in elem.attrib:
elem_data['Time'] = elem.attrib[attrib])
for child in elem.getchildren():
elem_data[getparent().attrib['name'])] = child.text
data.append(elem_data)
ndata = DataFrame(data)
最佳答案
我建议首先解析为 DataFrame,类似于您已经使用的方式(请参阅下文了解我的实现),然后根据您的要求对其进行调整。
那么你正在寻找一个pivot
:
In [11]: df
Out[11]:
child Time grandchild
0 blah 1200 100
1 blah 1300 30
2 abc 1200 2
3 abc 1300 4
4 abc 1400 2
In [12]: df.pivot('Time', 'child', 'grandchild')
Out[12]:
child abc blah
Time
1200 2 100
1300 4 30
1400 2 NaN
我先推荐parse from a file并将你想要的东西取出到元组列表中:
from lxml import etree
root = etree.parse(file_name)
parents = root.getchildren()[0].getchildren()
In [21]: elems = [(p.attrib['name'], int(c.attrib['Time']), int(gc.text))
for p in parents
for c in p
for gc in c]
In [22]: elems
Out[22]:
[('blah', 1200, 100),
('blah', 1300, 30),
('blah', 1400, 70),
('abc', 1200, 2),
('abc', 1300, 4),
('abc', 1400, 2)]
对于多个文件,您可以在更长的列表理解中对其进行处理。 除非你有大量的 xml(这里 files
是 xml 的列表),否则应该不会太慢...
elems = [(p.attrib['name'], int(c.attrib['Time']), int(gc.text))
for f in files
for p in etree.parse(f).getchildren()[0].getchildren()
for c in p
for gc in c]
将它们放入 DataFrame 中:
In [23]: pd.DataFrame(elems, columns=['child', 'Time', 'grandchild'])
Out[23]:
child Time grandchild
0 blah 1200 100
1 blah 1300 30
2 blah 1400 70
3 abc 1200 2
4 abc 1300 4
5 abc 1400 2
然后做枢轴。 :)
关于python - 将 XML 提取到以父属性作为列标题的数据框中,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/16991691/