python - celery 错过了心跳(on_node_lost)

标签 python django celery

我刚刚升级到 celery 3.1,现在我在我的日志中看到了这个::

on_node_lost - INFO - missed heartbeat from celery@queue_name for every queue/worker in my cluster.

根据文档,BROKER_HEARTBEAT 默认关闭,我还没有配置它。

我应该明确设置 BROKER_HEARTBEAT=0 还是应该检查其他内容?

最佳答案

Celery 3.1 添加了新的 mingle 和 gossip 程序。我也得到了很多错过的心跳和传递 --without-gossip 给我的 worker 清除了它。

https://docs.celeryproject.org/en/3.1/whatsnew-3.1.html#mingle-worker-synchronization

Mingle: Worker synchronization

The worker will now attempt to synchronize with other workers in the same cluster.

Synchronized data currently includes revoked tasks and logical clock.

This only happens at startup and causes a one second startup delay to collect broadcast responses from other workers.

You can disable this bootstep using the --without-mingle argument.

https://docs.celeryproject.org/en/3.1/whatsnew-3.1.html#gossip-worker-worker-communication

Gossip: Worker <-> Worker communication

Workers are now passively subscribing to worker related events like heartbeats.

This means that a worker knows what other workers are doing and can detect if they go offline. Currently this is only used for clock synchronization, but there are many possibilities for future additions and you can write extensions that take advantage of this already.

Some ideas include consensus protocols, reroute task to best worker (based on resource usage or data locality) or restarting workers when they crash.

We believe that although this is a small addition, it opens amazing possibilities.

You can disable this bootstep using the --without-gossip argument.

关于python - celery 错过了心跳(on_node_lost),我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/21132240/

相关文章:

python - 如何在 matplotlib/Python 中更改后端

python - 通过cmd启动识别正在运行的python程序

python - 如何在 Django REST 框架 api 中进行用户注册

python - 如何为 celery 和 prometheus 生成自定义应用程序指标/监控

python - 检查在 Celery 中完成的多个任务的最佳方法?

python - 计算类别所占的百分比

python - 如何在 Latex pygal 标签中渲染

django - 保存 Django FormWizard

python - 使用哪个 REST 动词?

django - 使用 Django REST 框架进行批量插入的最佳设计模式是什么?