Regex search for word anywhere within the string

Question

I am trying to add an additional "field" to the fluent-bit cri parser, however my regex never works. This is the original working regex: ^(?<time>[^ ]+) (?<stream>stdout|stderr) (?<logtag>[^ ]*) (?<message>.*)$

As it seems this looks sequentially through the log for the various elements. I now want to also add a "field" called level, like this: <level>, which is the log level and should be any of INFO, WARNING and ERROR. The level will occur anywhere within the message.

As an example: https://rubular.com/r/j6KxU3LrE3jjcA

Has anybody got an idea how I can add it?

Answer 1

Looks like your system is using Onigmo Regex in Ruby mode based on it's documentation.

EXAMPLE

I think this pattern should work nicely for you:

/^(?<time>[^ ]+) (?<stream>stdout|stderr) (?<logtag>[^ ]*)(?=(?:.* (?<level>(?:INFO|WARNING|ERROR))(?: |$))?) (?<message>.*)$/m

Based on your example you might use below without flags and delimiters:

^(?<time>[^ ]+) (?<stream>stdout|stderr) (?<logtag>[^ ]*)(?=(?:.* (?<level>(?:INFO|WARNING|ERROR))(?: |$))?) (?<message>.*)$

I have tested the above regex pattern in Ruby and below are the results with the sample test set. It will match in any order and making sure it is valid. In PCRE you will not get a group if not matched, but in Ruby you will get a nil value for the group if there is no match.

["2018-02-13T07:59:37.505201169-05:00", "stdout", "F", "ERROR", "ERROR"]
["2018-02-13T07:59:37.505201169-05:00", "stdout", "F", "ERROR", "line 1 user=INFO id=foo ERROR"]
["2018-02-13T07:59:37.505201169-05:00", "stdout", "F", "WARNING", "line 1 user=A id=foo WARNING"]
["2018-02-13T07:59:37.505201169-05:00", "stdout", "F", "WARNING", "WARNING line 1 user=A id=foo"]
["2018-02-13T07:59:37.505201169-05:00", "stdout", "F", nil, "line 1 user=WARNING id=foo"]
["2018-02-13T07:59:37.505201169-05:00", "stdout", "F", "WARNING", "INFO line 1 user=WARNING id=foo WARNING"]
["2018-02-13T07:59:37.505201169-05:00", "stdout", "F", "INFO", "INFO line 1 user=WARNING id=foo"]
["2018-02-13T07:59:37.505201169-05:00", "stdout", "F", "INFO", "line 1 user=ERROR id=foo INFO"]
["2018-02-13T07:59:37.505201169-05:00", "stdout", "F", nil, "line 1 user=ERROR id=foo"]
["2018-02-13T07:59:37.505201169-05:00", "stdout", "F", nil, "line 1 WARNINGERRORINFO user=ERROR id=foo"]
["2018-02-13T08:00:17.666563371-05:00", "stdout", "F", "INFO", "line 2 user= stdout INFO F id=bar"]
["2018-02-13T08:00:17.666563371-05:00", "stdout", "F", nil, "line 2 user= stdout F id=bar"]