python - 对 Django 应用程序的 celery 和 redis 进行故障排除

Question

我需要帮助解决 celery 使用 redis 运行 Django 应用程序的问题。在这个应用程序中，用户上传照片，我正在尝试运行一个后台进程，该进程将根据用户投票和上传后耗时定期对所有照片进行排名(将其视为类似 reddit 的基本排名算法)。

我的项目文件夹中有 celery.py(与 settings.py 相同的文件夹)，包含:

from __future__ import absolute_import
import os
from celery import Celery
from django.conf import settings

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'myproject.settings')

app = Celery('myapp', broker='redis://localhost:6379/0', backend='redis://localhost:6379/0',include=['myfolder.myapp.tasks'])
app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS) 

app.conf.update(
    CELERY_TASK_RESULT_EXPIRES=3600,
)

if __name__ == '__main__':
    app.start()

上面文件中提到的

tasks.py 位于 myapp 文件夹中，包含:

import os
from myproject import celery_app1
import time
from myapp.models import Photo

@celery_app1.task(name='tasks.rank_all_photos')
def rank_all_photos():
    for photo in Photo.objects.all():
        photo.set_rank()
        print "ranked"

myproject 文件夹中的

__init__.py 包含:

from __future__ import absolute_import
from .celery import app as celery_app1

最后，settings.py中的相关配置如下:

import djcelery
djcelery.setup_loader()

BROKER_URL = 'redis://localhost:6379/0'

CELERY_IMPORTS = ('myapp.tasks', )  

CELERY_ALWAYS_EAGER = False

CELERY_RESULT_BACKEND = 'redis://localhost:6379/0'

CELERY_ACCEPT_CONTENT = ['json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_IGNORE_RESULT=True

from datetime import timedelta

CELERYBEAT_SCHEDULE = {
    'tasks.rank_all_photos': {
        'task': 'tasks.rank_all_photos',
        'schedule': timedelta(seconds=10),
    },
}

CELERY_TIMEZONE = 'UTC'

请注意，'djcelery' 也包含在 INSTALLED_APPS 中。对我来说，以上所有内容看起来都是正确的。当我运行 celery worker -A myproject --loglevel=INFO 时，我看到输出:

---- **** ----- 
--- * ***  * -- Linux-3.16.0-30-generic-x86_64-with-Ubuntu-14.04-trusty
-- * - **** --- 
- ** ---------- [config]
- ** ---------- .> app:         myapp:0x7f0a15acb310
- ** ---------- .> transport:   redis://localhost:6379/0
- ** ---------- .> results:     redis://localhost:6379/0
- *** --- * --- .> concurrency: 2 (prefork)
-- ******* ---- 
--- ***** ----- [queues]
 -------------- .> celery           exchange=celery(direct) key=celery


[tasks]
  . tasks.rank_all_photos

[2016-05-17 02:19:19,733: INFO/MainProcess] Connected to redis://localhost:6379/0
[2016-05-17 02:19:19,745: INFO/MainProcess] mingle: searching for neighbors
[2016-05-17 02:19:20,750: INFO/MainProcess] mingle: all alone
/home/hassan/.virtualenvs/myenv/local/lib/python2.7/site-packages/celery/fixups/django.py:265: UserWarning: Using settings.DEBUG leads to a memory leak, never use this setting in production environments!
  warnings.warn('Using settings.DEBUG leads to a memory leak, never '

[2016-05-17 02:19:20,761: WARNING/MainProcess] /home/hassan/.virtualenvs/myenv/local/lib/python2.7/site-packages/celery/fixups/django.py:265: UserWarning: Using settings.DEBUG leads to a memory leak, never use this setting in production environments!
  warnings.warn('Using settings.DEBUG leads to a memory leak, never '

[2016-05-17 02:19:20,761: WARNING/MainProcess] celery@hassan ready.

我看到了该输出，但我没有看到任何证据表明 tasks.py 已被处理。不打印任何输出，也不进行任何排名。

我是初学者，因此可能缺少一些基本知识。你能帮我看看并解决这个问题吗？我目前只是试图在本地的 fg 中测试它——一旦它启动并运行，我将研究在生产中对其进行守护进程。谢谢。

Answer 1

您还需要运行 celery beat。

celery beat -A myproject --loglevel=INFO