python - 查找同级元素的文本，其中原始元素与特定字符串匹配

Question

我想从一堆 html 表格中提取一些数据价格。表格包含各种价格，当然表格数据标签不包含任何有用的东西。

<div id="item-price-data">
  <table>
    <tbody>
      <tr>
        <td class="some-class">Normal Price:</td>
        <td class="another-class">$100.00</td>
      </tr>
      <tr>
        <td class="some-class">Member Price:</td>
        <td class="another-class">$90.00</td>
      </tr>
      <tr>
        <td class="some-class">Sale Price:</td>
        <td class="another-class">$80.00</td>
      </tr>
      <tr>
        <td class="some-class">You save:</td>
        <td class="another-class">$20.00</td>
      </tr>
    </tbody>
  </table>
</div>

我唯一关心的是那些与文本为“正常价格”的元素配对的价格。

我想做的是扫描表的后代，找到 <td>具有该文本的标签，然后从它的同级标签中提取文本。

我遇到的问题是在 BeautifulSoup 中 descendants属性返回 NavigableString 的列表, 不是 Tag .

如果我这样做:

from bs4 import BeautifulSoup
from urllib import request

html = request.urlopen(url)
soup = BeautifulSoup(html, 'lxml')

div = soup.find('div', {'id': 'item-price-data'})
table_data = div.find_all('td')

for element in table_data:
    if element.get_text() == 'Normal Price:':
        price = element.next_sibling

print(price)

我一无所获。有没有简单的方法来取回字符串值？

Answer 1

您可以使用 find_next()方法你也可能需要一些正则表达式:

演示:

>>> import re
>>> from bs4 import BeautifulSoup
>>> html = """<div id="item-price-data">
...   <table>
...     <tbody>
...       <tr>
...         <td class="some-class">Normal Price:</td>
...         <td class="another-class">$100.00</td>
...       </tr>
...       <tr>
...         <td class="some-class">Member Price:</td>
...         <td class="another-class">$90.00</td>
...       </tr>
...       <tr>
...         <td class="some-class">Sale Price:</td>
...         <td class="another-class">$80.00</td>
...       </tr>
...       <tr>
...         <td class="some-class">You save:</td>
...         <td class="another-class">$20.00</td>
...       </tr>
...     </tbody>
...   </table>
... </div>"""
>>> soup = BeautifulSoup(html, 'lxml')
>>> div = soup.find('div', {'id': 'item-price-data'})
>>> for element in div.find_all('td', text=re.compile('Normal Price')):
...     price = element.find_next('td')
...     print(price)
... 
<td class="another-class">$100.00</td>

如果您不想将正则表达式引入其中，那么以下内容对您有用。

>>> table_data = div.find_all('td')
>>> for element in table_data:
...     if 'Normal Price' in element.get_text():
...         price = element.find_next('td')
...         print(price)
... 
<td class="another-class">$100.00</td>