python - 如何使用Python BeautifulSoup提取td HTML标签？

Question

我正在尝试废弃 webpage并从中提取前缀及其名称。但是，对于某些标签，我无法提取它们，我的猜测是存在不可见的标签。这是我的Python代码:

opener.addheaders = [('User-agent', 'Mozilla/5.0')]
response = opener.open('http://bgp.he.net/AS23028#_prefixes')
html = response.read()
soup = BeautifulSoup(html)
soup_1 = soup.find("table", id = "table_prefixes4")
soup_2 = soup_1.findAll("td")
print soup_2

有人知道如何获取标签后的名称吗？以下是该页面的 html 内容:

<div class="flag alignright floatright"><img alt="United States" src="/images/flags/us.gif?1282328089" title="United States"/></div>
</td>, <td class="nowrap">
<a href="/net/209.176.111.0/24">209.176.111.0/24</a>
</td>, <td>Savvis

我想从 HTML 中提取前缀“209.176.111.0/24”和“Savvis”

Answer 1

数据就在那里；页面中没有丢失任何内容。 HTML 似乎没有被破坏(足以)导致标签丢失，也没有任何 JavaScript 改变浏览器中的页面:

for row in soup.select('table#table_prefixes4 tr'):
    print row.get_text(' - ', strip=True)

打印整个表格，包括标题。

仅获取单元格:

for row in soup.select('table#table_prefixes4 tr'):
    cells = row.find_all('td')
    if not cells:
        continue
    print [cell.get_text(strip=True) for cell in cells]

后者产生:

>>> for row in soup.select('table#table_prefixes4 tr'):
...     cells = row.find_all('td')
...     if not cells:
...         continue
...     print [cell.get_text(strip=True) for cell in cells]
... 
[u'38.229.0.0/16', u'PSINet, Inc.']
[u'38.229.0.0/19', u'PSINet, Inc.']
[u'38.229.32.0/19', u'PSINet, Inc.']
[u'38.229.64.0/19', u'PSINet, Inc.']
[u'38.229.128.0/17', u'PSINet, Inc.']
[u'38.229.252.0/22', u'PSINet, Inc.']
[u'68.22.187.0/24', u'AS23028.NET']
[u'192.138.226.0/24', u'Computer Systems Consulting Services']
[u'203.28.18.0/24', u'Information Technology Services']
[u'204.74.64.0/24', u'SAUNET']
[u'209.176.111.0/24', u'Savvis']
[u'216.90.108.0/24', u'Savvis']