python - 如何用Python高效下载文件？

Question

我想下载文件。

我使用了asyncio，但下载文件时长时间没有响应。此外，当服务器证书无效时也会出现问题。

这是我的代码:

socket.setdefaulttimeout(30)
async def get_file(url, folder, session):
    filename = folder.strip('\\') + '\\' + str(datetime.datetime.now()).replace(':', '_') + '.pdf'
    async with session.get(url, headers = headers) as resp:
        if resp.status == 200:
            f = await aiofiles.open(filename, mode='wb')
            await f.write(await resp.read())
            await f.close()

async def download_pdf(urls, folder):
    connector = aiohttp.TCPConnector(limit=60)
    semaphore = asyncio.Semaphore(10)
    async with semaphore:
        async with ClientSession(connector=connector, headers=headers) as session:
            tasks = [get_file(url, folder, session) for url in urls]
            result = await asyncio.gather(*tasks)

urls = ['https://www.example.com/abc.pdf', 'https://www.example2.com/def.pdf']
directory = r'C:\data'
asyncio.run(download_pdf(urls, directory))

如何下载文件高效？

Answer 1

使用 requests 库，您可以将响应流式传输到文件中:

with open(r"/path/to/file", 'w+b') as f:
    for chunk in requests.get(yourURL, stream=True).iter_content(chunk_size=1024):
        f.write(chunk)