elasticsearch - Elasticsearch查询在Logstsah中失败:期望冒号分隔字段名称和值

Question

我一直在尝试使用logstash查询elasticsearch事件，以便可以用更多人类可读数据填充某些字段。我有一个人名索引，在索引其他类型的事件时，我会尝试使用该索引来增强其他字段。

我查询字符串有问题，但是我不知道是什么。我只是不断收到此警告和person_query_failed标记:

[2019-10-03T10:38:52,759][WARN ][logstash.filters.elasticsearch] Failed to query elasticsearch for previous event {:index=>"logstash-people", :error=>"Unexpected character ('}' (code 125)): was expecting a colon to separate field name and value\n at [Source: (byte[])\"{\n \"query\": {\n \"query_string\": { \"query\": { \"id:109\" } }\n },\n \"_source\": { \"cn\" }\n}\"; line: 3, column: 42]"}

[2019-10-03T10:38:52,760][WARN ][logstash.filters.elasticsearch] Failed to query elasticsearch for previous event {:index=>"logstash-people", :error=>"Unexpected character ('}' (code 125)): was expecting a colon to separate field name and value\n at [Source: (byte[])\"{\n \"query\": {\n \"query_string\": { \"query\": { \"id:4\" } }\n },\n \"_source\": { \"cn\" }\n}\"; line: 3, column: 40]"}

[2019-10-03T10:38:52,764][WARN ][logstash.filters.elasticsearch] Failed to query elasticsearch for previous event {:index=>"logstash-people", :error=>"Unexpected character ('}' (code 125)): was expecting a colon to separate field name and value\n at [Source: (byte[])\"{\n \"query\": {\n \"query_string\": { \"query\": { \"id:49\" } }\n },\n \"_source\": { \"cn\" }\n}\"; line: 3, column: 41]"}

这是我的查询模板person_query.json:

{ 
  "query": { 
    "query_string": { "query": { "id:%{[dn_people]}" } }
  },
  "_source": { "cn" } 
}

这是我的logstash过滤器配置:

filter {
    elasticsearch {
        hosts => [ "https://localhost:9200" ]
        index => "logstash-people"
        query_template => "/path/to/person_query.json"
        fields => { "cn" => "person" }
        user => admin
        password => password
        ca_file => "/etc/elasticsearch/root-ca.pem"
        tag_on_failure => person_query_failed
    }
}

查看警告日志时，至少查询字符串本身似乎可以正常运行:“id:％{[dn_people]}”转换为\“id:109 \”，\“id:4 \”和\“id:125 \“。

在编写查询时，我一直在看这个:https://discuss.elastic.co/t/how-to-query-elasticsearch-and-take-some-fields-from-old-data/75300/3

有人可以帮助并向正确的方向推我吗？

我正在将Opendistro用于Elasticsearch。

Answer 1

我认为问题出在这里

Unexpected character ('}' (code 125)): was expecting a colon to separate field name and value\n at [Source: (byte[])\"{\n \"query\": {\n \"query_string\": { \"query\": { \"id:49\" } }\n },\n \"_source\": { \"cn\" }\n}\"; line: 3, column: 41]"}

您的查询模板应如下所示:

{
  "query": { 
    "query_string": { "query": { "id": %{[dn_people]} } }
  },
  "_source": { "cn" } 
}

区别在于，您需要通过引用整个字段名称，后跟冒号和值来分隔字段和值。

您可以通过查看已解决的查询来确定问题:

\"id:109\"

没有转义字符，则如下所示:

"id:109"

Elasticsearch将整个字符串解释为字段名称，并且找不到以下冒号和值。

它必须解决:

\"id\":109 or unescaped: "id":109