我正在使用该代码来抓取客户评论。一切都按照我希望代码执行的操作进行,但我无法正确获取评级的类或属性,因此代码始终返回 Ratings
的空白结果。柱子。
有人可以帮我找到正确的属性并修复 Ratings
代码行?
from bs4 import BeautifulSoup
import requests
import pandas as pd
import json
print ('all imported successfuly')
# Initialize an empty dataframe
df = pd.DataFrame()
for x in range(1, 37):
names = []
headers = []
bodies = []
ratings = []
published = []
updated = []
reported = []
link = (f'https://www.trustpilot.com/review/fabfitfun.com?page={x}')
print (link)
req = requests.get(link)
content = req.content
soup = BeautifulSoup(content, "lxml")
articles = soup.find_all('article', {'class':'review'})
for article in articles:
names.append(article.find('div', attrs={'class': 'consumer-information__name'}).text.strip())
headers.append(article.find('h2', attrs={'class':'review-content__title'}).text.strip())
try:
bodies.append(article.find('p', attrs={'class':'review-content__text'}).text.strip())
except:
bodies.append('')
try:
#ratings.append(article.find('div', attrs={'class':'star-rating star-rating--medium'}).text.strip())
ratings.append(article.find('div', attrs={'class': 'star-rating star-rating--medium'})['alt'])
except:
ratings.append('')
dateElements = article.find('div', attrs={'class':'review-content-header__dates'}).text.strip()
jsonData = json.loads(dateElements)
published.append(jsonData['publishedDate'])
updated.append(jsonData['updatedDate'])
reported.append(jsonData['reportedDate'])
# Create your temporary dataframe of the first iteration, then append that into your "final" dataframe
temp_df = pd.DataFrame({'User Name': names, 'Header': headers, 'Body': bodies, 'Rating': ratings, 'Published Date': published, 'Updated Date':updated, 'Reported Date':reported})
df = df.append(temp_df, sort=False).reset_index(drop=True)
print ('pass1')
df.to_csv('FabfitfunReviews007.csv', index=False, encoding='utf-8')
print ('excel done')
最佳答案
只需更改代码中的这一行:
ratings.append(article.find_all("img", alt=True)[0]["alt"])
df.Rating
然后输出到:0 1 star: Bad
1 5 stars: Excellent
2 5 stars: Excellent
3 5 stars: Excellent
4 5 stars: Excellent
5 5 stars: Excellent
6 5 stars: Excellent
似乎更容易找到
img
标记文章并从中检索替代文本。
关于python - 网页抓取空白返回 - 错误的元素,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/60566258/