javascript - 如何使用 Selenium 和 Python 抓取传单 map 多边形

Question

这篇文章与这篇文章非常相似:Using selenium and python to extract data when it pops up after mouse hover
但我找不到我想要的答案。
我正在尝试抓取一张与此非常相似的传单 map :https://leafletjs.com/examples/choropleth/ ，理想情况下，我想下载将鼠标移到多边形上后出现的所有信息:
原始帖子在每个圆形元素上循环，我想对每个多边形做同样的事情。
代码试验:

from selenium.common.exceptions import TimeoutException
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
driver = webdriver.Chrome
driver.get("https://leafletjs.com/examples/choropleth/")
timeout = 1000

explicit_wait30 = WebDriverWait(driver, 30)
try:
    # Wait for all circles to load
    poli = explicit_wait30.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, '.leaflet-interactive')))
except TimeoutException:
    driver.refresh()


data = []
i=1
for circle in poli:
    i+=1
    # Execute mouseover on the element
    driver.execute_script("const mouseoverEvent = new Event('mouseover');arguments[0].dispatchEvent(mouseoverEvent)", poli)
    # Wait for the data to appear
    listing = explicit_wait30.until(EC.visibility_of_element_located((By.CSS_SELECTOR, '#listingHover')))
    data.append(listing.text)
    # Close the listing
    driver.execute_script("arguments[0].click()", listing.find_element_by_tag_name('button'))
    print(i)
    if i>15 : 
        break

我得到错误:

JavascriptException: Message: javascript error: arguments[0].dispatchEvent is not a function
  (Session info: chrome=85.0.4183.102)

似乎“传单交互式”元素没有事件类型的鼠标悬停，如何重现将鼠标移到多边形上的人类 Action ？

Answer 1

抓取传单 map 并提取将鼠标移到多边形上后出现的所有信息，因为所需的元素在 <iframe> 内所以你必须:

诱导 WebDriverWait使所需的框架可用并切换到它。

诱导 WebDriverWait对于所需的 visibility_of_element_located。

您可以使用以下 Locator Strategies :

driver.get('https://leafletjs.com/examples/choropleth/')
driver.execute_script("return arguments[0].scrollIntoView(true);", WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//h2[text()='Interactive Choropleth Map']"))))
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe[src='example.html']")))
elements = driver.find_elements_by_css_selector("svg.leaflet-zoom-animated>g path")
for element in elements:
    ActionChains(driver).move_to_element(element).perform()
    print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[@class='info leaflet-control']"))).text)

备注 :您必须添加以下导入:

from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

控制台输出:

US Population Density
Alabama
94.65 people / mi2
US Population Density
Hover over a state
US Population Density
Arizona
57.05 people / mi2
US Population Density
Arkansas
56.43 people / mi2
US Population Density
California
241.7 people / mi2
US Population Density
Colorado
49.33 people / mi2
US Population Density
Connecticut
739.1 people / mi2
US Population Density
Delaware
464.3 people / mi2
US Population Density
Maryland
596.3 people / mi2
US Population Density
Hover over a state
US Population Density
Georgia
169.5 people / mi2
US Population Density
Hover over a state
US Population Density
Montana
6.858 people / mi2
US Population Density
Illinois
231.5 people / mi2
US Population Density
Indiana
181.7 people / mi2
US Population Density
Iowa
54.81 people / mi2
US Population Density
Kansas
35.09 people / mi2
US Population Density
Kentucky
110 people / mi2
US Population Density
Mississippi
63.5 people / mi2
US Population Density
Maine
43.04 people / mi2
US Population Density
Virginia
204.5 people / mi2
US Population Density
Massachusetts
840.2 people / mi2
US Population Density

引用
您可以在以下位置找到一些相关的讨论:

Ways to deal with #document under iframe

Switch to an iframe through Selenium and python