我已经实现了一个多进程下载器。 如何打印可以自动刷新的状态栏(完成率,下载速度) 在终端的不同部分。
像这样:
499712 [6.79%] 68k/s // keep refreshing
122712 [16.79%] 42k/s // different process/thread
99712 [56.32%] 10k/s
代码:
download(...)
...
f = open(tmp_file_path, 'wb')
print "Downloading: %s Bytes: %s" % (self.file_name, self.file_size)
file_size_dl = 0
block_sz = 8192
start_time = time.time()
while True:
buffer = self.opening.read(block_sz)
if not buffer:
break
file_size_dl += len(buffer)
f.write(buffer)
end_time = time.time()
cost_time = end_time - start_time
if cost_time == 0:
cost_time = 1
status = "\r%10d [%3.2f%%] %3dk/s" % (file_size_dl,
file_size_dl * 100. / self.file_size,
file_size_dl * 100. / 1024 / 1024 / cost_time)
print status,
sys.stdout.flush()
f.close()
DownloadProcess继承Process类并触发下载方法。
我使用队列来存储 url。下面是启动过程
...
for i in range(3):
t = DownloadProcess(queue)
t.start()
for url in urls:
queue.put(url)
queue.join()
最佳答案
下面是一个同时实现了多处理和多线程的demo。要尝试其中一个,只需取消注释代码顶部的导入行。如果您在单行上有一个进度条,那么您可以使用打印 '\r' 的技术将光标移回该行的开头。但是如果你想拥有多行进度条,那么你将不得不变得更花哨一些。每次我想打印进度条时,我只是清空屏幕。查看文章 console output on Unix in Python它对我生成下面的代码有很大帮助。他展示了这两种技术。您还可以试试 curses 库,它是 python 标准库的一部分。问题Multiline progress bars问类似的事情。主线程/进程生成执行工作的子线程,并使用队列将它们的进度传回主线程。我强烈建议使用队列进行进程间/线程间通信。然后主线程显示进度并等待所有子线程结束执行,然后退出。
代码
import time, random, sys, collections
from multiprocessing import Process as Task, Queue
#from threading import Thread as Task
#from Queue import Queue
def download(status, filename):
count = random.randint(5, 30)
for i in range(count):
status.put([filename, (i+1.0)/count])
time.sleep(0.1)
def print_progress(progress):
sys.stdout.write('\033[2J\033[H') #clear screen
for filename, percent in progress.items():
bar = ('=' * int(percent * 20)).ljust(20)
percent = int(percent * 100)
sys.stdout.write("%s [%s] %s%%\n" % (filename, bar, percent))
sys.stdout.flush()
def main():
status = Queue()
progress = collections.OrderedDict()
workers = []
for filename in ['test1.txt', 'test2.txt', 'test3.txt']:
child = Task(target=download, args=(status, filename))
child.start()
workers.append(child)
progress[filename] = 0.0
while any(i.is_alive() for i in workers):
time.sleep(0.1)
while not status.empty():
filename, percent = status.get()
progress[filename] = percent
print_progress(progress)
print 'all downloads complete'
main()
演示
关于python - Python中多处理或多线程的动态刷新打印,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/13944959/