python - 如何调试 "pthread_cond_wait: Resource busy"？

Question

我编写了一个使用 python 多线程库执行 API 调用的脚本。它大大加快了处理速度，因为瓶颈是网络，而不是我主机上的任何东西(输入某人说 python 在这里不执行真正的多线程)。

问题是，有时当我运行脚本时，我会收到此错误，我的脚本最终会挂起/ sleep :

pthread_cond_wait: Resource busy

我不知道如何弄清楚为什么会发生这种情况。如何获得更多上下文来调试问题？我是否需要将打印语句放在一堆随机位置并希望捕获导致此问题的任何问题？有没有更好的调试方法？

如果有帮助，这就是我实现多线程的方式:

for i in range(threads): # make the threads
        t = threading.Thread(target=queue_worker, args=[apikey, q, retries, hit_threshold]) # The threads will use the "queue_worker" function with these parameters
        t.daemon = True
        t.start() # start the thread!
# Data is put onto the queue and queue_worker does the API work here...
...
q.join() # Clean up and close the threads when the threads are all idle (no more data on the queue)

编辑:

queue_worker，api和主要代码基本上是这样的:

def queue_worker(apikey, q, retries, hit_threshold)
   api_data = q.get()
   for x in range(retries)
      try:
         response = do_api(api_data, apikey)
      except Exception as error:
         time.sleep(5)
         continue
   else:
      error_count = error_count + 1
      q.task_done()
      continue
   #... data parsing code here...
   #... printing parsed data to screen here if a particular value returned is greater than "hit_threshold"...
   q.task_done()

def do_api(api_data, apikey)
   params = { 'apikey': apikey, 'resource': api_data }
   response = requests.get('https://MYURL.com/api', params=params, timeout=10)
   return response

if __name__ == '__main__':
   threads = 50
   q = Queue.Queue(threads)
   for i in range(threads): # make the threads
      t = threading.Thread(target=queue_worker, args=[apikey, q, retries, hit_threshold]) # The threads will use the "queue_worker" function with these parameters
      t.daemon = True
      t.start() # start the thread!
   # Data is put onto the queue and queue_worker does the API work here...
   ...
   q.join() # Clean up and close the threads when the threads are all idle (no more data on the queue)

Answer 1

Comment: Any tips on debugging?

1. 使用自己的锁、条件或其他线程函数来仔细检查您的代码，以便嵌套使用。
2. 使用自己的锁来访问共享变量。

3. Read Python Threads and the Global Interpreter Lock and try this "work around".
   There are other ways to accelerate the GIL manipulation or avoid it:

   • call ''time.sleep()'' - set ''sys.setcheckinterval()'' - run Python in optimized mode - dump process-intensive tasks into C-extensions - use the subprocess module to execute commands

<小时/>

很可能，您面临的是 Python GIL!

what-is-a-global-interpreter-lock-gil

其他线程之一拥有锁。
锁定的使用不一致。