docker - 流利的输入编解码器的Logstash无法正常工作

Question

我已经在使用带Gelf的Logstash了，想检查流利的输入(主要是由于基于TCP的Docker Log-Driver流利，而不是仅基于UDP的Gelf)。我的测试配置是这样的:

input {
  gelf {
    port => 12345
  }
  tcp {
    codec => fluent
    port => 23456
  }
}

filter {
}

output {
  stdout { codec => rubydebug { metadata => true } }
}

我可以使用以下方式发送Gelf日志:

docker run -it \
           --log-driver gelf \
           --log-opt gelf-address=udp://localhost:12345 \
           --log-opt tag=gelf-test \
        ubuntu:16.04 /bin/bash -c 'echo $(date -u +"%Y-%m-%dT%H:%M:%SZ") Hello gelf'

但是流利版本不起作用:

docker run -it \
           --log-driver fluentd \
           --log-opt fluentd-address=localhost:23456 \
           --log-opt tag=fluent-test \
        ubuntu:16.04 /bin/bash -c 'echo $(date -u +"%Y-%m-%dT%H:%M:%SZ") Hello fluent'

我可以验证logstash正在接收输入:

echo 'Hello TCP' | nc localhost 23456

An error occurred. Closing connection {:client=>"172.17.0.1:42012", :exception=>#, :backtrace=>["org/jruby/RubyTime.java:1073:in at'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-event-2.4.0-java/lib/logstash/timestamp.rb:32:inat'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-codec-fluent-2.0.4-java/lib/logstash/codecs/fluent.rb:41:in decode'", "org/msgpack/jruby/MessagePackLibrary.java:195:ineach'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-codec-fluent-2.0.4-java/lib/logstash/codecs/fluent.rb:40:in decode'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-tcp-3.0.6/lib/logstash/inputs/tcp.rb:153:inhandle_socket'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-tcp-3.0.6/lib/logstash/inputs/tcp.rb:143:in `server_connection_thread'"], :level=>:error}

我还禁用了流畅的编解码器，并在那里正确发送了流畅的日志和logstash错误，并按预期将流畅的msgpack解析为常规TCP事件的message字段。

Received an event that has a different character encoding than you configured. {:text=>"\x94\xABfluent-test\xD2X¢鄣log\xD9\\"2017-03-10T12:58:17Z Hello fluent\r\xACcontainer_id\xD9@9cbd13eb83a02a1a4d4f83ff063d4e40b4419b7dcbcef960e4689495caa5c132\xAEcontainer_name\xAF/ecstatic_kilby\xA6source\xA6stdout\xC0", :expected_charset=>"UTF-8", :level=>:warn}

{
       "message" => "\\x94\\xABfluent-test\\xD2X¢鄣log\\xD9\\\"2017-03-10T12:58:17Z Hello fluent\\r\\xACcontainer_id\\xD9@9cbd13eb83a02a1a4d4f83ff063d4e40b4419b7dcbcef960e4689495caa5c132\\xAEcontainer_name\\xAF/ecstatic_kilby\\xA6source\\xA6stdout\\xC0",
      "@version" => "1",
    "@timestamp" => "2017-03-10T12:58:18.069Z",
          "host" => "172.17.0.1",
          "port" => 42016
}

我没有其他想法，是否有人遇到过这个问题，或者对进一步调试有任何想法？

Answer 1

您是否可以尝试Fluentd实例？，这样更容易确定问题所在。快速浏览似乎Logstash Fluent编解码器无法正常工作。