我有一个使用 celery 执行异步任务的应用程序,我使用 redis 作为其代理和结果后端,我将 redis 设置为使用 unix 套接字。 这是我的 celery 和经纪人的网址
brok = 'redis+socket://:ABc@/tmp/redis.sock'
app = Celery('NTWBT', backend=brok, broker=brok)
app.conf.update(
BROKER_URL=brok,
BROKER_TRANSPORT_OPTIONS={
"visibility_timeout": 3600
},
CELERY_RESULT_BACKEND=brok,
CELERY_ACCEPT_CONTENT=['pickle', 'json', 'msgpack', 'yaml'],
)
但每次我添加作业 celery 都会给我这个错误
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/celery/app/trace.py", line 283, in trace_task
uuid, retval, SUCCESS, request=task_request,
File "/usr/local/lib/python2.7/dist-packages/celery/backends/base.py", line 257, in store_result
request=request, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/celery/backends/base.py", line 491, in _store_result
self.set(self.get_key_for_task(task_id), self.encode(meta))
File "/usr/local/lib/python2.7/dist-packages/celery/backends/redis.py", line 160, in set
return self.ensure(self._set, (key, value), **retry_policy)
File "/usr/local/lib/python2.7/dist-packages/celery/backends/redis.py", line 149, in ensure
**retry_policy
File "/usr/local/lib/python2.7/dist-packages/kombu/utils/__init__.py", line 246, in retry_over_time
return fun(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/celery/backends/redis.py", line 169, in _set
pipe.execute()
File "/usr/local/lib/python2.7/dist-packages/redis/client.py", line 2620, in execute
self.shard_hint)
File "/usr/local/lib/python2.7/dist-packages/redis/connection.py", line 897, in get_connection
connection = self.make_connection()
File "/usr/local/lib/python2.7/dist-packages/redis/connection.py", line 906, in make_connection
return self.connection_class(**self.connection_kwargs)
TypeError: __init__() got an unexpected keyword argument 'socket_connect_timeout'
我应该使用哪个选项让 celery 不为其 redis 连接设置超时?
最佳答案
看来这个问题与你系统安装的redis-server版本有关,socket_connect_timeout最早是在redis 2.10.0引入的。
所以你需要更新你的redis版本。
如果您在 ubuntu 服务器上运行,您可以安装官方 apt 存储库:
$ sudo apt-get install -y python-software-properties
$ sudo add-apt-repository -y ppa:rwky/redis
$ sudo apt-get update
$ sudo apt-get install -y redis-server
并更新到最新版本的 celery。
这是 celery 中的 github 问题,因为不仅你会遇到这个问题:https://github.com/celery/celery/issues/2903
如果一切都不适合您,我建议使用 rabbitmq 而不是 Redis:
$ sudo apt-get install rabbitmq-server
$ sudo pip install librabbitmq
并在您的应用中使用此 CELERY_BROKER_URL 配置 celery :
'amqp://guest:guest@localhost:5672//'
我希望这个答案能满足您的所有需求。 干杯
关于redis - 带有redis的 celery - unix超时,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/34307784/