Logstash - Parsing Optional Logs through Custom grok pattern

Question

Following is my log patterns

Sample 1 :
2022-06-14 02:03:22.051  INFO  [ServiceName,TraceId,SpanID] 109171 --- [Thread] ClassName : A=ValueA B=ValueB C=ValueC

Sample 2:
2022-06-14 02:03:22.051  INFO  [ServiceName,TraceId,SpanID] 109171 --- [Thread] ClassName : D=ValueD B=ValueB C=ValueC

Sample 3:
2022-06-14 02:03:22.051  INFO  [ServiceName,TraceId,SpanID] 109171 --- [Thread] ClassName : D=ValueD E=ValueE C=ValueC F=ValueF

Sample 4:
INFO  [ServiceName,TraceId,SpanID] 109171 --- [Thread] ClassName : Some Log Message

Following is the grok pattern I tried

Custom Patterns: A A=.*A B B=.*B and similar

For Example following log pattern :

%{TIMESTAMP_ISO8601:timestamp}?%{SPACE}*%{LOGLEVEL:log-level}%{SPACE}*\[%{DATA:service},%{DATA:ZTraceId},%{DATA:ZSpanId}\]%{SPACE}*%{NUMBER:ProcessId}%{GREEDYDATA:message}%{A:Afield}

Above log works in Sample1 but not 2,3 &4

%{TIMESTAMP_ISO8601:timestamp}? ---- Optional way works in all samples

At the same time, %{A:Afield}? Doesnt work in Sample 1,2,3,4....In Sample Afield is not identified

Can anyone please provide a solution for optional custom grok pattern

Answer 1

You can try this grok pattern which might help

%{TIMESTAMP_ISO8601:timestamp}?%{SPACE}*%{DATA:log-level}?%{SPACE}*\[%{DATA:service}\,%{DATA:ZTraceId}\,%{DATA:ZSpanId}\] %{NUMBER:ProcessId} --- \[%{DATA:buglevel}\] %{DATA:class}: %{GREEDYDATA:message} %{GREEDYDATA:message} %{GREEDYDATA:message}