python - 在以下输入字段上使用 Scrapy 和 send_keys

Question

我试图专注于 twitter.com/login 和 send_keys('foo') 上的电子邮件输入，但是我没有运气。有人可以告诉我为什么我的实现是错误的:

import time

from scrapy.spider import BaseSpider
from scrapy.selector import HtmlXPathSelector

from selenium import webdriver 

class MySpider(BaseSpider):
    name = "new"
    allowed_domains = ["twitter.com"]
    start_urls = ["https://www.twitter.com/login"]

    def __init__(self):
    self.driver = webdriver.Firefox()

    def parse(self, response):
    self.driver.get(response.url)
    time.sleep(1)

    #print response.body
    username = self.driver.find_element_by_class_name('email-input')
    username.click()
    username.clear()
    username.send_keys('foo')


    #self.driver.close()

如有任何帮助，我们将不胜感激。另外，当我尝试这样做时，我收到以下错误:

2015-08-22 15:44:09-0500 [scrapy] DEBUG: Telnet console listening on 127.0.0.1:6023 2015-08-22 15:44:10-0500 [new] DEBUG: Redirecting (301) to https://twitter.com/login> from https://www.twitter.com/login> 2015-08-22 15:44:10-0500 [new] DEBUG: Crawled (200) https://twitter.com/login> (referer: None) 2015-08-22 15:44:22-0500 [new] ERROR: Spider error processing https://twitter.com/login> (referer: None) Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/twisted/internet/base.py", line 1201, in mainLoop self.runUntilCurrent() File "/usr/lib/python2.7/dist-packages/twisted/internet/base.py", line 824, in runUntilCurrent call.func(*call.args, **call.kw) File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 382, in callback self._startRunCallbacks(result) File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 490, in _startRunCallbacks self._runCallbacks() --- --- File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 577, in _runCallbacks current.result = callback(current.result, *args, **kw) File "/home/tyrick/Documents/twitter_followers/twitter_followers/spiders/newUsers.py", line 22, in parse username.click() File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webelement.py", line 65, in click self._execute(Command.CLICK_ELEMENT) File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webelement.py", line 385, in _execute return self._parent.execute(command, params) File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 173, in execute self.error_handler.check_response(response) File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 166, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.ElementNotVisibleException: Message: Element is not currently visible and so may not be interacted with Stacktrace: at fxdriver.preconditions.visible (file:///tmp/tmpHPD9xT/extensions/fxdriver@googlecode.com/components/command-processor.js:8959:12) at DelayedCommand.prototype.checkPreconditions_ (file:///tmp/tmpHPD9xT/extensions/fxdriver@googlecode.com/components/command-processor.js:11618:15) at DelayedCommand.prototype.executeInternal_/h (file:///tmp/tmpHPD9xT/extensions/fxdriver@googlecode.com/components/command-processor.js:11635:11) at DelayedCommand.prototype.executeInternal_ (file:///tmp/tmpHPD9xT/extensions/fxdriver@googlecode.com/components/command-processor.js:11640:7) at DelayedCommand.prototype.execute/< (file:///tmp/tmpHPD9xT/extensions/fxdriver@googlecode.com/components/command-processor.js:11582:5)

Answer 1

问题是，有多个具有 email-input 类的元素，而实际定位的第一个元素是不可见的。您需要页面容器中的一个:

self.driver.find_element_by_css_selector("#page-container .email-input")