Python subprocess() 等到一系列命令中的最后一个命令完成

Question

我的部分代码如下所示。 for 循环将启动一堆我希望它们并行运行的 shell 命令。但是，在这个 for 循环之后，我有一些命令只有当且仅当在 for 循环期间触发的所有这些进程都完成时才应该运行。有没有办法做到这一点？如果我对每个进程都使用 wait()，那么这些进程将简单地按顺序运行，而不是以并行方式运行。

#split the bedgraph file by chromName
bedfile_list = (options.filename).split(',')
bed1,bed2 = bedfile_list[0],bedfile_list[1]
subgenome_name_list = (options.subgenome_name).split(',')
sub1,sub2 = subgenome_name_list[0],subgenome_name_list[1]
for chromosome in chrom:
    #os.system('grep \'{}\' {} > {}.{}.temp'.format(chromosome, bed1, sub1, chromosome))
    #os.system('grep \'{}\' {} > {}.{}.temp'.format(chromosome, bed2, sub2, chromosome))
    p1 = subprocess.Popen('grep \'{}\' {} > {}.{}.temp'.format(chromosome, bed1, sub1, chromosome),shell=True)
    p2 = subprocess.Popen('grep \'{}\' {} > {}.{}.temp'.format(chromosome, bed2, sub2, chromosome),shell=True)



subprocess.Popen('rm *.temp')

Answer 1

当然!只需存储您的 popen 对象，然后您就可以在继续之前检查它们是否全部完成:

# create an empty list to add all of the popen objects to
processes = []

for chromosome in chrom:
    p1 = subprocess.Popen('grep \'{}\' {} > {}.{}.temp'.format(chromosome, bed1, sub1, chromosome),shell=True)
    p2 = subprocess.Popen('grep \'{}\' {} > {}.{}.temp'.format(chromosome, bed2, sub2, chromosome),shell=True)

    # stash the popen objects for later use
    processes.append(p1)
    processes.append(p2)

# before moving on, call wait() on all of the objects to ensure they're done
# this is a blocking call, so the loop won't complete until all processes have returned
for p in processes:
    p.wait()

# now do your post processing work