为了下载文件，我创建了一个 urlopen 对象(urllib2 类)并分 block 读取它。
我想多次连接到服务器并在六个不同的 session 中下载文件。这样做，下载速度应该会更快。许多下载管理器都有此功能。
我考虑指定我想在每个 session 中下载的文件部分，并以某种方式同时处理所有 session 。我不确定如何实现这一目标。
最佳答案
关于运行并行请求，您可能希望使用 urllib3或 requests .
我花了一些时间来列出类似的问题:
寻找 [python] +download +concurrent 给出了这些有趣的:
- Concurrent downloads - Python
- What is the fastest way to send 100,000 HTTP requests in Python?
- Library or tool to download multiple files in parallell
- Download multiple pages concurrently?
- Python: simple async download of url content?
- Python, gevent, urllib2.urlopen.read(), download accelerator
- Python/Urllib2/Threading: Single download thread faster than multiple download threads. Why?
- Scraping landing pages of a list of domains
- A clean, lightweight alternative to Python's twisted?
寻找 [python] +http +concurrent 给出这些:
- Python: How to make multiple HTTP POST queries in one moment?
- Multi threaded web scraper using urlretrieve on a cookie-enabled site
寻找[python] +urllib2 +slow:
- Python urllib2.open is slow, need a better way to read several urls
- Python 2.6: parallel parsing with urllib2
- How can I speed up fetching pages with urllib2 in python?
- Threading HTTP requests (with proxies)
寻找[python] +download +many:
关于python - 并行获取文件，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/9007456/