elasticsearch - Multiline logstash "next" not grouping -
short: having troubles multiline. tag "multiline" on log doesn't put them together.
explanation: logs receive
september 22nd 2016, 13:43:52.738 [0m[[31merror[0m] [0mtotal time: 368 s, completed 2016-09-22 13:43:52[0m september 22nd 2016, 13:43:51.738 [0m[[0minfo[0m] [0m[36msuites: completed 29, aborted 0[0m[0m september 22nd 2016, 13:43:51.738 [0m[[31merror[0m] [0mfailed: total 100, failed 4, errors 0, passed 96[0m september 22nd 2016, 13:43:51.737 [0m[[0minfo[0m] [0m[36mrun completed in 1 minute, 24 seconds.[0m[0m september 22nd 2016, 13:43:51.737 [0m[[0minfo[0m] [0mscalatest[0m
the line "total time: %{number} s" repeated multiple time , i'm interested in these total time coming after "total, failed, error" line. between first , second line none or several logs.
my configuration is:
grok { #1 match => {"message" => "\[.m\[\u001b\[3.m%{notspace:level}\u001b\[0m\] \u001b\[0m%{notspace:status}: total %{number}, failed %{number}, errors %{number}$ add_tag => [ "test.continue" ] tag_on_failure => [] } #2 if "test.continue" in [tags]{ multiline { pattern => "%{timestamp_iso8601}\u001b\[0m$" => "next" negate => true } } #3 #overalltime grok { match => {"message" => "\[.m\[\u001b\[3.m%{notspace:level}\u001b\[0m\] \u001b\[0mtotal time: %{number:seconds:int} s, completed"} add_tag => [ "test.overalltime" ] tag_on_failure => [] }
what is:
beats_input_codec_plain_applied, test.continue, multiline [0m[[31merror[0m] [0mfailed: total 100, failed 4, errors 0, passed 96[0m
the first log gets multiline tag , test.continue doesn't behave expect.
the logic understand is:
- if find
[0m[[31merror[0m] [0mfailed: total 100, failed 4, errors 0, passed 96[0m
put tag "test.continue", - multiline every log tag "test.continue" , send next line find until find log end
%{timestamp_iso8601}\u001b\[0m$
- extract time log
more explanation:
i'm believing behaviour be, not happening.
1 finding trigger [0m[[31merror[0m] [0mfailed: total 100, failed 4, errors 0, passed 96[0m
2 once finds it, take , added @ beginning of next line. since first part of log still match, add again tag , sends multiline again [0m[[31merror[0m] [0mfailed: total 100, failed 4, errors 0, passed 96[0m [0m[[0minfo[0m] [0m[36msuites: completed 29, aborted 0[0m[0m
3 have first, second, ... , until finds log finishes timestamp , breaks multiline. next log not added tag.
[0m[[31merror[0m] [0mfailed: total 100, failed 4, errors 0, passed 96[0m [0m[[0minfo[0m] [0m[36msuites: completed 29, aborted 0[0m[0m [0m[[31merror[0m] [0mtotal time: 368 s, completed 2016-09-22 13:43:52[0m
Comments
Post a Comment