我是Logtash的新手!
我配置好了,到目前为止一切正常。
我的日志文件如下:
2014-04-27 16:24:43 DEBUG b45e66 T+561 10.31.166.155 /v1/XXX<!session> XXX requested for category_ids: only_pro: XXX_ids:14525
如果我使用以下conf文件:
input { file { path => "/logs/*_log" }} output { elasticsearch { host => localhost } }
它将在ES中放置以下内容:
{
_index: "logstash-2014.04.28",
_type: "logs",
_id: "WIoUbIvCQOqnz4tMZzMohg",
_score: 1,
_source: {
message: "2014-04-27 16:24:43 DEBUG b45e66 T+561 10.31.166.155 This is my log !",
@version: "1",
@timestamp: "2014-04-28T14:25:52.165Z",
host: "MYCOMPUTER",
path: "\logs\xxx_app.log"
}
}
我如何在我的日志中获取字符串,以使整个文本不会在_source.message上?
例如:我希望我可以将其解析为:
{
_index: "logstash-2014.04.28",
_type: "logs",
_id: "WIoUbIvCQOqnz4tMZzMohg",
_score: 1,
_source: {
logLevel: "DEBUG",
messageId: "b45e66",
sendFrom: "10.31.166.155",
logTimestamp: "2014-04-27 16:24:43",
message: "This is my log !",
@version: "1",
@timestamp: "2014-04-28T14:25:52.165Z",
host: "MYCOMPUTER",
path: "\logs\xxx_app.log"
}
}
最佳答案
您需要通过过滤器对其进行解析,例如grok filter。这可能有些棘手,所以请耐心尝试,尝试,尝试。并查看预定义的patterns。
您的消息的开始将是%{DATESTAMP} %{WORD:logLevel} %{WORD:messageId} %{GREEDYDATA:someString} %{IP}
grokdebugger是一个非常有用的工具,可为您提供帮助。
完成后,您的配置应如下所示
input {
stdin {}
}
filter {
grok {
match => { "message" => "%{DATESTAMP} %{WORD:logLevel} %{WORD:messageId} %{GREEDYDATA:someString} %{IP}" }
}
}
output {
elasticsearch { host => localhost }
}
关于elasticsearch - 使用logstash和 Elasticsearch 进行自定义解析,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/23343878/