我正在从another question陷入这个问题。
我如下创建了管道。
{
"bool-pipeline": {
"description": "converts FALSE/TRUE to boolean",
"processors": [
{
"convert": {
"field": "secure_flag",
"type": "boolean",
"ignore_missing": true
}
}
]
}
}
我的有效值为FALSE,TRUE,[空白]
我能够检索管道。
我在Logstash配置中添加了以下内容。
input {
file {
path => "/Users/gibbs/Documents/search/mini_system.csv"
start_position => beginning
sincedb_path => "/dev/null"
}
}
filter {
csv {
columns => [
"secure_flag",
"mini_system_key",
"hw_contract_end_date"
]
separator => ","
}
mutate {
remove_field => ["path", "host"]
}
}
output {
stdout {
codec => rubydebug
}
elasticsearch {
action => "index"
hosts => ["127.0.0.1:9200"]
index => "mini_system"
document_id => "%{mini_system_key}"
pipeline => "%{bool-pipeline}"
}
}
但是,当我加载数据时,它将引发以下错误。
[2020-05-10T16:13:05,691][DEBUG][o.e.a.b.T.BulkRequestModifier] [gopir-mac-1] failed to execute pipeline [_none] for document [mini_system/_doc/50395971|1038832]
java.lang.IllegalArgumentException: pipeline with id [%{bool-pipeline}] does not exist
at org.elasticsearch.ingest.IngestService.executePipelines(IngestService.java:407) [elasticsearch-7.6.2.jar:7.6.2]
at org.elasticsearch.ingest.IngestService.access$000(IngestService.java:75) [elasticsearch-7.6.2.jar:7.6.2]
at org.elasticsearch.ingest.IngestService$3.doRun(IngestService.java:384) [elasticsearch-7.6.2.jar:7.6.2]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:692) [elasticsearch-7.6.2.jar:7.6.2]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.6.2.jar:7.6.2]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_231]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_231]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_231]
我无能为力。我正在尝试几个小时。
有什么建议吗?
当我检查日志时,在 flex 搜索日志中看到该错误。 Logstash有一个衬板。
最佳答案
您的摄取管道名称为bool-pipeline
,应在elasticsearch
中使用它,而不是%{bool-pipeline}
。%{FIELD}
用于与事件相关的值,这不是您的情况,使用%{bool-pipeline}
将使logstash尝试从事件中名为bool-pipeline
的字段中提取带有摄取管道名称的值,并且由于该字段不存在,因此得到这个错误。
关于elasticsearch - Logstash管道不存在,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/61712659/