我正在尝试从 Transfermarkt.com 中抓取(足球队)表。对于一个项目,但有些列具有相同的类名,无法区分。
列 [2,10] 具有独特的类并且工作正常。我正在努力寻找一种方法来获得休息。
from bs4 import BeautifulSoup
import pandas as pd
headers = {'User-Agent':
'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36'}
page = "https://www.transfermarkt.com/hertha-bsc-u17/kader/verein/21066/saison_id/2018/plus/1"
pageTree = requests.get(page, headers=headers)
pageSoup = BeautifulSoup(pageTree.content, 'html.parser')
Players = pageSoup.find_all("a", {"class": "spielprofil_tooltip"})
Values = pageSoup.find_all("td", {"class": "zentriert"})
PlayersList = []
ValuesList = []
for i in range(0, 25):
PlayersList.append(Players[i].text)
ValuesList.append(Values[i].text)
df = pd.DataFrame({"Players": PlayersList, "Values": ValuesList})
我想抓取该表行上的所有列。
最佳答案
我会得到所有<tr>
然后使用 for
循环获取所有<td>
在排队。然后我可以使用索引来获取列,并且可以使用不同的方法从列中获取值。
import requests
from bs4 import BeautifulSoup
import pandas as pd
data = {
'name': [],
'data of birth': [],
'height': [],
'foot': [],
'joined': [],
'contract until': [],
}
headers = {
'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36'
}
url = "https://www.transfermarkt.com/hertha-bsc-u17/kader/verein/21066/saison_id/2018/plus/1"
response = requests.get(url, headers=headers)
soup = BeautifulSoup(response.content, 'html.parser')
all_tr = soup.find_all('tr', {'class': ['odd', 'even']})
print('rows:', len(all_tr))
for row in all_tr:
all_td = row.find_all('td', recursive=False)
print('columns:', len(all_td))
for column in all_td:
print(' >', column.text)
data['name'].append( all_td[1].text.split('.')[0][:-1] )
data['data of birth'].append( all_td[2].text[:-5])
data['height'].append( all_td[4].text )
data['foot'].append( all_td[5].text )
data['joined'].append( all_td[6].text )
data['contract until'].append( all_td[8].text )
df = pd.DataFrame(data)
print(df.head())
结果:
name data of birth height foot joined contract until
0 Kilian Schubert Sep 9, 2002 1,80 m right Jul 1, 2018 -
1 Raphael Bartell Jan 26, 2002 1,82 m - Jul 1, 2018 -
2 Till Aufderheide Jun 15, 2002 1,79 m - Jul 1, 2018 -
3 Milan Kremenovic Mar 8, 2002 1,91 m - Jul 1, 2018 30.06.2020
4 Adnan Alagic Jul 4, 2002 1,86 m right Jul 1, 2018 30.06.2021
关于python - 如何抓取单元格值具有相同类名的网站表格?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/55992681/