elasticsearch - 当Elasticsearch死/无法写入磁盘时的Logstash可用性

标签 elasticsearch logstash rsyslog

我有rsyslog通过TCP将日志转发到logstash。如果logstash不可用,则rsyslog将建立队列。

如果有logstash可用,但elasticsearch已死或由于某种原因而无法写入文件系统。

是否有logstash拒绝其他TCP消息的方法。

谢谢

最佳答案

根据life of an event description:

An output can fail or have problems because of some downstream cause, such as full disk, permissions problems, temporary network failures, or service outages. Most outputs should keep retrying to ship any events that were involved in the failure.

If an output is failing, the output thread will wait until this output is healthy again and able to successfully send the message. Therefore, the output queue will stop being read from by this output and will eventually fill up with events and block new events from being written to this queue.

A full output queue means filters will block trying to write to the output queue. Because filters will be stuck, blocked writing to the output queue, they will stop reading from the filter queue which will eventually cause the filter queue (input -> filter) to fill up.

A full filter queue will cause inputs to block when writing to the filters. This will cause each input to block, causing each input to stop processing new data from wherever that input is getting new events.


这意味着,如果elasticsearch输出开始失败,则整个管道将被阻塞,这正是您所需要的。您看到不同的地方了吗?

关于elasticsearch - 当Elasticsearch死/无法写入磁盘时的Logstash可用性,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/23284711/

相关文章:

elasticsearch - 是否可以通过Logstash中的先前值更改字段

logstash - 为什么 Logstash 每次重新启动时都会导入相同的文件?

linux - 设置 rsyslog 过滤器

ubuntu-12.04 - Ubuntu 12.04 记录器不会写入远程系统日志

elasticsearch - sbt 无法解析来自 sonatype 快照的依赖关系

elasticsearch - ElasticSearch从所有文档的字段中获取所有术语

spring-boot - 需要为ElasticsearchRepository findBy查询编写自定义分析器

elasticsearch - Logstash Web界面仍然可用吗?

ubuntu - 将特定日志从 systemd 服务重定向到单独的文件不起作用

node.js - 弹性对象内部的Elasticsearch搜索数组