python - Celery 4 不自动发现任务

Question

我有一个 Django 1.11 和 Celery 4.1 项目，我已经根据 setup docs 配置了它。 .我的 celery_init.py 看起来像

from __future__ import absolute_import

import os

from celery import Celery

# set the default Django settings module for the 'celery' program.
os.environ['DJANGO_SETTINGS_MODULE'] = 'myproject.settings.settings'

app = Celery('myproject')

app.config_from_object('django.conf:settings', namespace='CELERY')

#app.autodiscover_tasks(lambda: settings.INSTALLED_APPS) # does nothing
app.autodiscover_tasks() # also does nothing

print('Registering debug task...')
@app.task(bind=True)
def debug_task(self):
    print('Request: {0!r}'.format(self.request))

但是，当我启动一个 worker 时:

.env/bin/celery worker -A myproject -l info

它显示除了示例“debug_task”之外没有找到任何任务，即使我安装了几个带有 Celery 任务的应用程序，应该通过调用 app.autodiscover_task() 找到。这是我的工作人员生成的初始输出:

 -------------- celery@localhost v4.1.0 (latentcall)
---- **** ----- 
--- * ***  * -- Linux-4.13.0-16-generic-x86_64-with-Ubuntu-16.04-xenial 2017-10-31 15:56:42
-- * - **** --- 
- ** ---------- [config]
- ** ---------- .> app:         myproject:0x7f952856d650
- ** ---------- .> transport:   amqp://guest:**@localhost:5672//
- ** ---------- .> results:     amqp://
- *** --- * --- .> concurrency: 4 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** ----- 
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery


[tasks]
  . myproject.celery_init.debug_task

[2017-10-31 15:56:42,180: INFO/MainProcess] Connected to amqp://guest:**@127.0.0.1:5672//
[2017-10-31 15:56:42,188: INFO/MainProcess] mingle: searching for neighbors
[2017-10-31 15:56:43,211: INFO/MainProcess] mingle: all alone
[2017-10-31 15:56:43,237: INFO/MainProcess] celery@localhost ready.

我的应用 tasks.py 文件中的所有遗留任务定义如下:

from celery.task import task

@task(name='mytask')
def mytask():
    blah

文档建议使用 shared_task 装饰器，所以我尝试了:

from celery import shared_task

@shared_task
def mytask():
    blah

但是我的 Celery worker 仍然没有看到它。我做错了什么？

编辑:我已经能够通过在我的设置的 CELERY_IMPORTS 列表中明确列出它们来让任务显示出来，但即使那样我也必须大量编辑 tasks.py 删除我的 Django 项目(models.py 等)的所有导入，否则它会引发异常 Apps aren't loaded yet. 这总比没有好，但需要大量重构。有没有更好的办法？

Answer 1

我遇到了类似的问题，解决方案是将 include kwarg 添加到您的 celery 调用中。

The include argument is a list of modules to import when the worker starts. You need to add our tasks module here so that the worker is able to find our tasks.

app = Celery('myproject', 
             backend = settings.CELERY.get('backend'),
             broker = settings.CELERY.get('broker'),
             include = ['ingest.tasks.web', ... ])

查看 http://docs.celeryproject.org/en/latest/getting-started/next-steps.html#proj-celery-py了解更多信息