python - 多线程模块比 python 中的多处理慢

Question

我发现对于同一任务，线程模块比多重处理花费更多的时间。

import time
import threading
import multiprocessing

def func():
    result = 0
    for i in xrange(10**8):
        result += i

num_jobs = 10


# 1. measure, how long it takes 10 consistent func() executions 
t = time.time()
for _ in xrange(num_jobs):
    func()
print ("10 consistent func executions takes:\n{:.2f} seconds\n".format(time.time() - t))        

# 2. threading module, 10 jobs
jobs = []
for _ in xrange(num_jobs):
    jobs.append(threading.Thread(target = func, args=()))
t = time.time()
for job in jobs:
    job.start()
for job in jobs:
    job.join()      
print ("10 func executions in parallel (threading module) takes:\n{:.2f} seconds\n".format(time.time() - t))        

# 3. multiprocessing module, 10 jobs
jobs = []
for _ in xrange(num_jobs):
    jobs.append(multiprocessing.Process(target = func, args=()))
t = time.time()
for job in jobs:
    job.start()
for job in jobs:
    job.join()
print ("10  func executions in parallel (multiprocessing module) takes:\n{:.2f} seconds\n".format(time.time() - t))  

结果:

10 consistent func executions takes: 25.66 seconds

10 func executions in parallel (threading module) takes: 46.00 seconds

10 func executions in parallel (multiprocessing module) takes: 7.92 seconds

1) 为什么使用多处理模块的实现效果比使用线程模块的效果更好？

2) 为什么一致函数执行比使用线程模块花费的时间更少？

Answer 1

您的两个问题都可以通过 docs 的摘录得到解答。 :

The mechanism used by the CPython interpreter to assure that only one thread executes Python bytecode at a time. This simplifies the CPython implementation by making the object model (including critical built-in types such as dict) implicitly safe against concurrent access. Locking the entire interpreter makes it easier for the interpreter to be multi-threaded, at the expense of much of the parallelism afforded by multi-processor machines.

大胆强调我的。您最终会花费大量时间在线程之间切换以确保它们全部运行完成。这就像将一根绳子切成小块，然后将这些小块再次绑在一起，一次一根绳子。

多处理模块的结果符合预期，因为每个进程都在自己的地址空间中执行。这些进程除了处于同一进程组且具有相同的父进程外，彼此独立。