python - 如何使用 'contents'来抓取我想要的值？

Question

我正在关注这个link 从 website 中抓取数据

我想抓取姓名、网址、年份和国籍，当我尝试使用以下代码时

import requests
import csv
from bs4 import BeautifulSoup
import bs4


f = csv.writer(open('z_artist_names_assignment.csv', 'w'))
f.writerow(['N'])

pages = []

for i in range(1, 2):
    url = 'https://web.archive.org/web/20121007172955/https://www.nga.gov/collection/anZ' + str(i) + '.htm'
    pages.append(url)



for item in pages:
    page = requests.get(item,timeout=10)
    soup = BeautifulSoup(page.text, 'html.parser')

    last_links = soup.find(class_='AlphaNav')
    last_links.decompose()

    artist_name_list = soup.find(class_='BodyText')
    artist_name_list_items = artist_name_list.find_all('a')

    nationality_list = soup.find(class_='BodyText')
    nationality_list_items = nationality_list.find_all('td')

    for artist_name in artist_name_list_items:
        names = artist_name.contents[0]
        links = 'https://web.archive.org' + artist_name.get('href')

    for nationality in nationality_list_items:
        nationality = nationality.contents[0]
        print(nationality)

返回打印(国籍) 不仅是内容，还有名称和选项卡，例如

<a href="/web/20121007172955/https://www.nga.gov/cgi-bin/tsearch?artistid=11630">Zabaglia, Niccola</a>
Italian, 1664 - 1750
<a href="/web/20121007172955/https://www.nga.gov/cgi-bin/tsearch?artistid=34202">Zaccone, Fabian</a>
American, 1910 - 1992
<a href="/web/20121007172955/https://www.nga.gov/cgi-bin/tsearch?artistid=3475">Zadkine, Ossip</a>
French, 1890 - 1967

我只想要“意大利，1664 - 1750”或“意大利”或“1664 - 1750”。 如何使用内容方法来获取这些值？

这是 HTML

<tr valign="top"><td><a href="/web/20121007172955/http://www.nga.gov/cgi-bin/tsearch?artistid=3452">Zalce, Alfredo</a></td><td>Mexican, born 1908</td></tr>

Answer 1

我认为最好找到所有包含艺术家信息的“tr”元素，而不是“td”。

下面是示例。希望对您有帮助!

entries = soup.find_all("tr", {"valign" : "top"})


links = ['https://web.archive.org{}'.format(entry.contents[0].a['href']) for entry in entries]
names = [entry.contents[0].text for entry in entries]
nationalities = [entry.contents[1] for entry in entries]