How do i set multiple key values into diskcache in single insert?

30 views Asked by At

I am trying to use the python diskcache to store the parse the log files and store the information into cache. As the data to parse is huge, rather than making entry into cache for every parsed record, I am trying to collect it in-memory and then dump the entire dict into diskcache for data persistence and to derive insights later. But i don't find any example/ sample to make insert of multiple key and value into diskcache.

The closest i could find is using transact.

    try:
        with cache_store.transact():
            for data_map in mappings_list:
                key = "{}{}".format(data_map[key_index],tags)
                cache_store[key] = data_map
    except Exception as ex:
        print('Key index:', key_index)

I follow the above method where the above mappings_list dict has data of size over 100k and this part of pushing data into diskcache gets very slow.

0

There are 0 answers