我正在尝试循环遍历一系列 URL 并从公司列表中抓取董事会成员。下面的循环似乎有问题,它只运行数组中的第一个元素并复制结果。任何对此的帮助将不胜感激。代码:
from bs4 import BeautifulSoup
import requests
#array of URLs to loop through, will be larger once I get the loop working correctly
tickers = ['http://www.reuters.com/finance/stocks/companyOfficers?symbol=AAPL.O', 'http://www.reuters.com/finance/stocks/companyOfficers?symbol=GOOG.O']
board_members = []
output = []
soup = BeautifulSoup(html, "html.parser")
for t in tickers:
html = requests.get(t).text
officer_table = soup.find('table', {"class" : "dataTable"})
for row in officer_table.find_all('tr'):
cols = row.find_all('td')
if len(cols) == 4:
board_members.append((t, cols[0].text.strip(), cols[1].text.strip(), cols[2].text.strip(), cols[3].text.strip()))
for t, name, age, year_joined, position in board_members:
output.append(('{} {:35} {} {} {}'.format(t, name, age, year_joined, position)))
最佳答案
soup = BeautifulSoup(html, "html.parser")
for t in tickers:
html = requests.get(t).text
officer_table = soup.find('table', {"class" : "dataTable"})
你把soup放在for循环之外,这会导致错误,因为当你使用BeautifulSoup(html, "html.parser")时,'html'不存在
只需在分配 html 后将其放入循环即可。
for t in tickers:
html = requests.get(t).text
soup = BeautifulSoup(html, "html.parser")
officer_table = soup.find('table', {"class" : "dataTable"})
关于python - BeautifulSoup 循环遍历 URL 数组,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/40903412/