python - 使用 Python(ElementTree？)解析非常大的 HTML 文件

Question

I asked about using BeautifulSoup to parse a very large (270MB) HTML file and getting a memory error andwas pointed toward ElementTree as a solution.

我正在尝试使用他们的事件驱动解析，documented here .使用较小的设置文件对其进行测试效果很好:

>>> settings = open('S:\\Documents\\FacebookData\\html\\settings.htm')
>>> for event, element in ET.iterparse(settings, events=("start", "end")):
    print("%5s, %4s, %s" % (event, element.tag, element.text))

成功打印出元素。然而，在实际编码过程开始之前，将相同的代码与“messages.htm”而不是“settings.htm”一起使用只是为了查看它是否工作，结果如下:

Traceback (most recent call last):
  File "<pyshell#16>", line 1, in <module>
    for event, element in ET.iterparse(source, events=("start", "end")):
  File "C:\Program Files (x86)\Python\lib\xml\etree\ElementTree.py", line 1294, in __next__
for event in self._parser.read_events():
  File "C:\Program Files (x86)\Python\lib\xml\etree\ElementTree.py", line 1277, in read_events
raise event
  File "C:\Program Files (x86)\Python\lib\xml\etree\ElementTree.py", line 1235, in feed
self._parser.feed(data)
  File "<string>", line None
xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 1, column 6

我想知道这是不是因为 ET 更适合解析 XML 文档？如果是这种情况，并且没有解决方法，那么我回到原点。非常感谢任何有关如何解析此文件以及如何调试的建议!

Answer 1

解析 HTML 或 XML 的一个很好的解决方案是 lxml 和 xpath。

要使用 xpath:

from lxml import etree
data = open('result.html','r').read()
doc = etree.HTML(data)

for tr in doc.xpath('//table/tr[@class="trmenu1"]'):
    print tr.xpath('./td/text()')