KV Processor for Filebeat

Question

Today our application is logging data in key value pairs, we need to push these log lines to logz.io by processing these key value pairs to fields. Today we are using filebeat to push log from application VM to logstash server and in logstash server we are using KV processor to process the message and push to logz.io.

Currently Filebeat does not support KV processor, do anyone know if this can be achieved in alternative way thus avoiding going through logstash server?

Answer 1

I am not that familiar with logz.io but if in the end, data is going to index in Elasticsearch, probably you can setup Elasticsearch ingest pipeline KV Processor.

Answer 2

You can contact logz.io support to customize your ingestion pipeline, and convert these pairs into fields.

Note: I work at logz.io