我正在编写一个脚本来下载一组文件。我成功地完成了这个并且它工作得很好。现在我尝试添加下载进度的动态打印输出。
对于小型下载(顺便说一下,它是 .mp4 文件),例如 5MB,进度效果很好并且文件成功关闭,从而生成完整且有效的下载 .mp4 文件。对于较大的文件,例如 250MB 及以上,它无法成功运行,我收到以下错误:
这是我的代码:
import urllib.request
import shutil
import os
import sys
import io
script_dir = os.path.dirname('C:/Users/Kenny/Desktop/')
rel_path = 'stupid_folder/video.mp4'
abs_file_path = os.path.join(script_dir, rel_path)
url = 'https://archive.org/download/SF145/SF145_512kb.mp4'
# Download the file from `url` and save it locally under `file_name`:
with urllib.request.urlopen(url) as response, open(abs_file_path, 'wb') as out_file:
eventID = 123456
resp = urllib.request.urlopen(url)
length = resp.getheader('content-length')
if length:
length = int(length)
blocksize = max(4096, length//100)
else:
blocksize = 1000000 # just made something up
# print(length, blocksize)
buf = io.BytesIO()
size = 0
while True:
buf1 = resp.read(blocksize)
if not buf1:
break
buf.write(buf1)
size += len(buf1)
if length:
print('\r[{:.1f}%] Downloading: {}'.format(size/length*100, eventID), end='')#print('\rDownloading: {:.1f}%'.format(size/length*100), end='')
print()
shutil.copyfileobj(response, out_file)
这对小文件非常有效,但对大文件我会出错。现在,如果我注释掉 progress 指示器代码,我不会收到更大文件的错误:
with urllib.request.urlopen(url) as response, open(abs_file_path, 'wb') as out_file:
# eventID = 123456
#
# resp = urllib.request.urlopen(url)
# length = resp.getheader('content-length')
# if length:
# length = int(length)
# blocksize = max(4096, length//100)
# else:
# blocksize = 1000000 # just made something up
#
# # print(length, blocksize)
#
# buf = io.BytesIO()
# size = 0
# while True:
# buf1 = resp.read(blocksize)
# if not buf1:
# break
# buf.write(buf1)
# size += len(buf1)
# if length:
# print('\r[{:.1f}%] Downloading: {}'.format(size/length*100, eventID), end='')#print('\rDownloading: {:.1f}%'.format(size/length*100), end='')
# print()
shutil.copyfileobj(response, out_file)
有人有什么想法吗?这是我项目的最后一部分,我真的很希望能够看到进展。再一次,这是 Python 3.5。感谢您提供的任何帮助!
最佳答案
您打开您的网址两次,一次作为response
,一次作为resp
。有了进度条,你就在消耗数据,所以当使用 copyfileobj
复制文件时,数据是空的(这可能是不准确的,因为它适用于小文件,但你正在做事情在这里两次,这可能是你问题的根源)
要获取进度条和有效文件,请执行以下操作:
with urllib.request.urlopen(url) as response, open(abs_file_path, 'wb') as out_file:
eventID = 123456
length = response.getheader('content-length')
if length:
length = int(length)
blocksize = max(4096, length//100)
else:
blocksize = 1000000 # just made something up
size = 0
while True:
buf1 = response.read(blocksize)
if not buf1:
break
out_file.write(buf1)
size += len(buf1)
if length:
print('\r[{:.1f}%] Downloading: {}'.format(size/length*100, eventID), end='')#print('\rDownloading: {:.1f}%'.format(size/length*100), end='')
print()
对您的代码进行的简化:
- 只有一个
urlopen
,作为response
- 没有
BytesIO
,直接写入out_file
关于python - 连接重置错误 : An existing connection was forcibly closed by the remote host,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/41110531/