我正在运行一个爬虫,它在 MAC 和 Ubuntu 系统上运行良好。 当我在 Windows 中运行它时,它会产生以下堆栈跟踪:
File "build\bdist.win32\egg\mechanize\_mechanize.py", line 203, in open
return self._mech_open(url, data, timeout=timeout)
File "build\bdist.win32\egg\mechanize\_mechanize.py", line 230, in _mech_open
response = UserAgentBase.open(self, request, data)
File "build\bdist.win32\egg\mechanize\_opener.py", line 193, in open
response = urlopen(self, req, data)
File "build\bdist.win32\egg\mechanize\_urllib2_fork.py", line 344, in _open
'_open', req)
File "build\bdist.win32\egg\mechanize\_urllib2_fork.py", line 332, in _call_chain
result = func(*args)
File "build\bdist.win32\egg\mechanize\_urllib2_fork.py", line 1170, in https_open
return self.do_open(conn_factory, req)
File "build\bdist.win32\egg\mechanize\_urllib2_fork.py", line 1118, in do_open
raise URLError(err)
URLError: <urlopen error [Errno 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond>
最佳答案
我非常怀疑问题与您的系统有关。要解决此问题,我建议:
- 检查您的 Mechanize 版本。旧版本可能会以不同方式处理请求。
- 调查
set_handle_robots
和addheaders
。问题可能与每个 IP 的请求过多有关。 (我怀疑这是真正的问题。) - 检查错误处理。正如上面的评论中所述,您无论如何都应该添加适当的错误处理。
如果上述解决方案均无效,请提供导致错误的代码。
关于Python Mechanize 连接失败问题,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/10631077/