下面是一个简单的例子,因为一个子进程没有返回任何东西就退出了,父进程一直在等待,所以卡住了。如果花费的时间太长,是否有办法使进程超时并让其余的继续?我是 python 多处理的初学者,我发现文档不是很有启发性。
import multiprocessing as mp
import time
def foo(x):
if x == 3:
sys.exit()
#some heavy computation here
return result
if __name__ == '__main__':
pool = mp.Pool(mp.cpu_count)
results = pool.map(foo, [1, 2, 3])
最佳答案
我遇到了同样的问题,我就是这样解决的。也许有更好的解决方案,但是,它也解决了未提及的问题。例如。如果该进程占用大量资源,则正常终止可能需要一段时间才能完成该进程——因此我使用强制终止 (kill -9
)。这部分可能仅适用于 Linux,因此如果您使用其他操作系统,您可能需要调整终止。
它是我自己代码的一部分,因此可能无法复制粘贴。
from multiprocessing import Process, Queue
import os
import time
timeout_s = 5000 # seconds after which you want to kill the process
queue = Queue() # results can be written in here, if you have return objects
p = Process(target=INTENSIVE_FUNCTION, args=(ARGS_TO_INTENSIVE_FUNCTION, queue))
p.start()
start_time = time.time()
check_interval_s = 5 # regularly check what the process is doing
kill_process = False
finished_work = False
while not kill_process and not finished_work:
time.sleep(check_interval_s)
now = time.time()
runtime = now - start_time
if not p.is_alive():
print("finished work")
finished_work = True
if runtime > timeout_s and not finished_work:
print("prepare killing process")
kill_process = True
if kill_process:
while p.is_alive():
# forcefully kill the process, because often (during heavvy computations) a graceful termination
# can be ignored by a process.
print(f"send SIGKILL signal to process because exceeding {timeout_s} seconds.")
os.system(f"kill -9 {p.pid}")
if p.is_alive():
time.sleep(check_interval_s)
else:
try:
p.join(60) # wait 60 seconds to join the process
RETURN_VALS = queue.get(timeout=60)
except Exception:
# This can happen if a process was killed for other reasons (such as out of memory)
print("Joining the process and receiving results failed, results are set as invalid.")
关于Python 多处理 : kill process if it is taking too long to return,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/59808975/