We have a custom log pipeline and currently, each step/service parses incoming json and modifies/adds tags. I was exploring options to reduce the cost of this serialization/de-serialization since some of the services don't modify data and just need to read/route the logs.
I was trying out flexbuffers (since the logs are from different services and don't share same schema).
I was thinking convert to flexbuffers::to_vec() and send this vector to next service. The downstream services can deserialize into other formats using flexbuffers::from_slice(). But I'm looking for ways where this vector can be directly used as flexbuffer::MapReader instead of going through intermediate steps.
fn test_flexbuffer_map() {
let log = serde_json::json!({"time": 123, "level": "INFO", "extra" : {"message" : "Hello! World"}});
let vec_fb = flexbuffers::to_vec(&log).unwrap();
let map_fb = ??;
let level = map_fb.idx("level"); // want to access fields directly
}
-- Update --
Not sure if this is correct way but this works.
let u8_fb: &[u8] = &vec_fb;
let map_fb = flexbuffers::Reader::get_root(u8_fb).unwrap().as_map();
let level = map_fb.idx("level").as_str();
println!("{level}");