我正在尝试确定网站列表中的网站是否为网上商店。
似乎大多数网络商店都有:
a
标签的href
中包含“cart”一词- 分配给类名称中包含“cart”一词的
li
标记
我相信我必须利用正则表达式,然后告诉 BeautifulSoup find
方法在 a
或 中搜索此正则表达式的 HTML 数据>li
标签。我该怎么办?
到目前为止,下面的代码在 HTML 数据中搜索 a
标记,其中 href
为 EXACTLY cart。
代码
import re
from bs4 import BeautifulSoup
from selenium import webdriver
websites = [
'https://www.nike.com/',
'https://www.youtube.com/',
'https://www.google.com/',
'https://www.amazon.com/',
'https://www.gamestop.com/'
]
shops = []
driver = webdriver.Chrome('chromedriver')
options = webdriver.ChromeOptions()
options.headless = True
options.add_argument('log-level=3')
with webdriver.Chrome(options=options) as driver:
for url in websites:
driver.get(url)
cart = re.compile('.*cart.*', re.IGNORECASE)
soup = BeautifulSoup(driver.page_source, 'html.parser')
if soup.find('a', href=cart):
shops.append(url)
print('\nSHOPS FOUND:')
for shop in shops:
print(shop)
输出:
SHOPS FOUND:
https://www.nike.com/
https://www.amazon.com/
最佳答案
您可以将 contains * 运算符与 css 属性选择器一起使用来指定 class 属性或 href 属性包含子字符串 cart。使用 Or 语法将两者(class 和 href)结合起来。 TODO:您可以考虑添加等待条件以确保所有 li
和 a
标记元素首先出现。
from bs4 import BeautifulSoup
from selenium import webdriver
websites = [
'https://www.nike.com/',
'https://www.youtube.com/',
'https://www.google.com/',
'https://www.amazon.com/',
'https://www.gamestop.com/'
]
shops = []
driver = webdriver.Chrome('chromedriver')
options = webdriver.ChromeOptions()
options.headless = True
options.add_argument('log-level=3')
with webdriver.Chrome(options=options) as driver:
for url in websites:
driver.get(url)
soup = BeautifulSoup(driver.page_source, 'html.parser')
items = soup.select('a[href*=cart], li[class*=cart]')
if len(items) > 0:
shops.append(url)
print('\nSHOPS FOUND:')
for shop in shops:
print(shop)
关于python - 确定网站是否是网上商店,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/55405590/