elasticsearch - 使用logstash解析json数组字符串

Question

我想使用logstash解析此json。

{"name":"bob","last":"builder", "atts":"{\"a\":111, \"b\":222}"}

{ "name" => "bob", "last" => "builder" "atts" => { "a" => 111, "b" => 222} }

Answer 1

两种选择!

使用Logstash解析JSON

如果要使用logstash-解析JSON，请在此处引用logstash插件:

https://www.elastic.co/guide/en/logstash/current/plugins-filters-json.html

要实现这一点，您将需要使用logstash.conf的过滤器部分:

filter {
   json {
     source => "message"
   }
}

该链接中还有更多json解码的示例。

使用Filebeat解析JSON

您的另一个选择是在文件拍子端解码json，然后将其转换为logstash。相关链接:

https://www.elastic.co/guide/en/beats/filebeat/current/decode-json-fields.html

https://discuss.elastic.co/t/parse-json-data-with-filebeat/80008/5

https://discuss.elastic.co/t/parse-json-data-with-filebeat/80008/7

https://discuss.elastic.co/t/how-to-read-json-file-using-filebeat-and-send-it-to-elasticsearch/91802

这是用于这种情况的样本filebeat.yml:

filebeat.inputs:
  - type: log
    paths:
      - 'path to the log directory you want to track'
    enter code here
    input_type: log
    json.keys_under_root: true
    json.add_error_key: true
    fields:
        log_type: 'type of log'

    processors:
    - decode_json_fields:
        fields: ["message"]
        process_array: true

    - add_tags:
        tags:
            - 'tag in elastic'

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml

setup.template.settings:
  index.number_of_shards: 1

output.logstash:
  # The Logstash hosts
  hosts: ["where logstash is running"]
  index: 'your index'

  codec.json:
    pretty: true
    escape_html: false

#================================ Processors =====================================
# Configure processors to enhance or manipulate events generated by the beat.
processors:
- decode_json_fields:
    fields: ["message"]
    process_array: true

json.keys_under_root: true
json.add_error_key: true

和

processors:
    - decode_json_fields:
        fields: ["message"]
        process_array: true

绝招。