python-3.x - 具有代理设置的 Selenium 抛出 UnexpectedAlertPresentException

Question

我将 Luminati 代理服务与 Selenium 一起用于抓取。我这样配置代理:

def get_driver(proxy_server):
    executable_path = os.path.join(app.config['JOBS_ROOT'], app.config['BROWSER_DRIVER'])
    firefox_profile = webdriver.FirefoxProfile()
    firefox_options = FirefoxOptions()
    firefox_options.add_argument('-headless')
    if proxy_server and proxy_server != LOCAL_HOST:
        firefox_profile.set_preference('network.proxy.type', 1)
        firefox_profile.set_preference('network.proxy.http', 'zproxy.lum-superproxy.io')
        firefox_profile.set_preference('network.proxy.http_port', 22225)
        firefox_profile.set_preference('network.proxy.ssl', 'zproxy.lum-superproxy.io')
        firefox_profile.set_preference('network.proxy.ssl_port', 22225)
    driver = webdriver.Firefox(
        executable_path=executable_path,
        firefox_options=firefox_options,
        firefox_profile=firefox_profile
    )
    return driver 

在这种配置下使用 Luminaty 需要身份验证:

我第一次尝试确保代理 IP 正常工作是使用 https://whatsmyip.org :

driver.get('https://whatsmyip.org')
alert = WebDriverWait(driver, 5).until(expected_conditions.alert_is_present())
alert.send_keys(app.config['LUMINATI_CUSTOMER'] + Keys.TAB + app.config['LUMINATI_PASSWORD'])
alert.accept()

但是，在使用 alert.accept() 接受后，驱动程序将关闭甚至不显示页面内容(另外，它似乎非常不稳定，有时有效，有时无效)。

所以我最终重复了 driver.get() 语句:

driver.get('https://whatsmyip.org')
alert = WebDriverWait(driver, 5).until(expected_conditions.alert_is_present())
alert.send_keys(app.config['LUMINATI_CUSTOMER'] + Keys.TAB + app.config['LUMINATI_PASSWORD'])
alert.accept()

driver.get('https://whatsmyip.org')

但我不认为它应该是这样工作的。

更重要的是，大多数时候我都会收到以下错误:

Traceback (most recent call last):
  File "/home/cesar/Development/manar/venv/lib/python3.7/site-packages/rq/worker.py", line 812, in perform_job
    rv = job.perform()
  File "/home/cesar/Development/manar/venv/lib/python3.7/site-packages/rq/job.py", line 588, in perform
    self._result = self._execute()
  File "/home/cesar/Development/manar/venv/lib/python3.7/site-packages/rq/job.py", line 594, in _execute
    return self.func(*self.args, **self.kwargs)
  File "./jobs/scrapping.py", line 185, in scrap_plate_number
    record = scrap_and_recognize(driver, vehicle)
  File "./jobs/scrapping.py", line 91, in scrap_and_recognize
    driver.find_element_by_xpath('//div[contains(@class, "jcrm-botondetalle")]/a').click()
  File "/home/cesar/Development/manar/venv/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 394, in find_element_by_xpath
    return self.find_element(by=By.XPATH, value=xpath)
  File "/home/cesar/Development/manar/venv/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 978, in find_element
    'value': value})['value']
  File "/home/cesar/Development/manar/venv/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/home/cesar/Development/manar/venv/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 241, in check_response
    raise exception_class(message, screen, stacktrace, alert_text)
selenium.common.exceptions.UnexpectedAlertPresentException: Alert Text: None
Message: Dismissed user prompt dialog: The proxy moz-proxy://zproxy.lum-superproxy.io:22225 is requesting a username and password. The site says: “Luminati”

我不知道为什么 Firefox 会关闭警报。有什么线索吗？

Answer 1

检查这个

driver.get("https://<username>:<password>@www.example.com/index.html")