Remove Avro type keys from JSON message format

Question

I'm trying to create a script to deserialize some Avro messages that comes from Kafka.

The messages have a format like:

{
  "value": {
    "value1": {
      "string": "AAAA"
    }
  }
}

and I need it to be something like that

{
  "value": {
    "value1":  "AAAA"
  }
}

Basically, delete that string.

I have schemas for both of them.

I need to move from the message that is serialized with a schema to a message that is deserialized with another schema.

I tried to do something with python avro/fastavro, but I didn't succed.

I can not just delete that and format because the Avro that I need to reformat are much more complex. So, I need something that will reformat these avros based on my schemas.

Answer 1

I can't tell from the question if you are trying to convert between schemas or just remove that string hint. If you are just trying to remove the hint, you can do this:

from fastavro import json_reader, json_writer

schema = {...}

with open('some-file', 'r') as fo:
    avro_reader = json_reader(fo, schema)
    records = [record for record in avro_reader]

with open('some-other-file', 'w') as out:
    json_writer(out, schema, records, write_union_type=False)