python - Tornado 错误: cannot switch to a different thread

Question

请求到达网络应用程序的tornado GET 处理程序。 从 GET 函数中，调用 blocking_task 函数。此 blocking_task 函数具有 @run_on_executor 装饰器。

但是这次执行失败了。 您能帮忙解决一下吗？看来 motor db 无法执行该线程。

import time
from concurrent.futures import ThreadPoolExecutor
from tornado import gen, web
from tornado.concurrent import run_on_executor
from tornado.ioloop import IOLoop
import argparse
from common.config import APIConfig
import sys
import os
import motor

parser = argparse.ArgumentParser()
parser.add_argument("-c", "--config-file", dest='config_file',
                    help="Config file location")
args = parser.parse_args()
CONF = APIConfig().parse(args.config_file)

client = motor.MotorClient(CONF.mongo_url)
db = client[CONF.mongo_dbname]
class Handler(web.RequestHandler):

    executor = ThreadPoolExecutor(10)

    def initialize(self):
        """ Prepares the database for the entire class """
        self.db = self.settings["db"]

    @gen.coroutine
    def get(self):
        self.blocking_task()

    @run_on_executor
    def blocking_task(self):
        mongo_dict = self.db.test_cases.find_one({"name": "Ping"})


if __name__ == "__main__":
    app = web.Application([
        (r"/", Handler),
    ],
        db=db,
        debug=CONF.api_debug_on,
    )
    app.listen(8888)
    IOLoop.current().start()




> ERROR:tornado.application:Exception in callback <functools.partial
> object at 0x7f72dfbe48e8> Traceback (most recent call last):   File
> "/usr/local/lib/python2.7/dist-packages/tornado-4.3-py2.7-linux-x86_64.egg/tornado/ioloop.py",
> line 600, in _run_callback
>     ret = callback()   File "/usr/local/lib/python2.7/dist-packages/tornado-4.3-py2.7-linux-x86_64.egg/tornado/stack_context.py",
> line 275, in null_wrapper
>     return fn(*args, **kwargs)   File "/usr/local/lib/python2.7/dist-packages/motor-0.5-py2.7.egg/motor/frameworks/tornado.py",
> line 231, in callback
>     child_gr.switch(future.result()) error: cannot switch to a different thread

您能帮忙解决一下吗？

Answer 1

来自docs

IOLoop and executor to be used are determined by the io_loop and executor attributes of self. To use different attributes, pass keyword arguments to the decorator

您必须提供一个初始化线程池执行器:

import time
from concurrent.futures import ThreadPoolExecutor
from tornado import gen, web
from tornado.concurrent import run_on_executor
from tornado.ioloop import IOLoop


class Handler(web.RequestHandler):

    executor = ThreadPoolExecutor(10)

    @gen.coroutine
    def get(self):
        self.blocking_task()

    @run_on_executor
    def blocking_task(self):
        time.sleep(10)


if __name__ == "__main__":
    app = web.Application([
        (r"/", Handler),
    ])
    app.listen(8888)
    IOLoop.current().start()

默认情况下，run_on_executor 在 executor 属性中搜索线程池，除非您显式传递其他属性，例如

_thread_pool = ThreadPoolExecutor(10)

@run_on_executor(executor='_thread_pool')
def blocking_task(self):
    pass

编辑

基本上 IOLoop 应该在单线程环境中使用(您可以在每个线程上运行单独的 IOLoop，但这不是您的情况)。要与 IOLoop 通信，您应该使用 add_callback ，这是唯一的线程安全函数。

你可以这样使用:

@run_on_executor
def blocking_task(self):
    IOLoop.instance().add_callback(some_update)

@gen.coroutine
def some_update():
    db.test_cases.update({ "name": "abc" }, { "$set": { "status" : "xyz" } } )

但是你真的需要线程吗？如果您在主 IOLoop 线程上安排更新，那么单独线程的目的是什么。