python - 有没有办法用 Popen.poll() 来区分未完成和阻塞

Question

我想“播种”一些进程，然后在完成后“收获”它们。使用 subprocess 模块，我在 subrun.py 中有以下代码:

import time, subprocess, shlex, os

ok = subprocess.Popen(shlex.split("python ok.py"),
                      stdout=subprocess.PIPE,
                      stderr=open(os.devnull, 'w'))


nok = subprocess.Popen(shlex.split("python nok.py"),
                       stdout=subprocess.PIPE,
                       stderr=open(os.devnull, 'w'))

procs = {'ok': ok, 'nok': nok}

while procs:
    running = procs.keys()
    print "running:", running
    for k in running:
        proc = procs[k]
        rc = proc.poll()
        if rc is None:
            pass  # still running
        else:
            del procs[k]
            print proc.stdout.read()

    time.sleep(.4)

ok.py如下

import sys
print "OK"
sys.exit(0)

和nok.py是

import sys
print "NOK" * 5000
sys.exit(0)

输出为

(dev) C:\work\dev\test>python subrun.py
running: ['ok', 'nok']
running: ['ok', 'nok']
OK

running: ['nok']
running: ['nok']
running: ['nok']
running: ['nok']
running: ['nok']
running: ['nok']
running: ['nok']
Traceback (most recent call last):
  File "subrun.py", line 27, in <module>
time.sleep(.4)

即。当子进程在 IO 上阻塞时，Popen.poll() 返回 None。

我可能可以为每个进程启动一个线程，这将调用 .communicate()[0]，但这似乎需要大量额外的记录......

有什么办法可以让它发挥作用吗？

Answer 1

如果你设置了stdout=PIPE，那么你应该读取管道，否则如果你的子进程生成足够的输出，它可能会永远阻塞。

这是您的代码中的一个错误。修复它。

<小时/>

要在子进程完成后获取所有输出:

#!/usr/bin/env python
import shlex
from multiprocessing.dummy import Pool
from subprocess import check_output

cmds = map(shlex.split, ["python ok.py", "python nok.py"])
outputs = Pool(len(cmds)).map(check_output, cmds)