peek in python persistqueue

12 views Asked by At

I am implementing a FIFOSQLiteQueue from the module persistqueue in python.

I have two threads: one gathering data and adding it to the queue and one dequeing and processing the data in the queue.

However I want my dequeue function to only remove the item from the queue after processing instead of before processing to prevent data loss if the progress crashes during processing the data.

Can anyone help me with this?

this is the code I currently use for creating the queue, enqueuing, and dequeueing

from persistqueue import FIFOSQLiteQueue
from sys import path

max_queue_size = 3
current_working_dir = f"{path[0]}"
queue_path  = f"{current_working_dir}/queue"
queue = FIFOSQLiteQueue(queue_path, multithreading=True, auto_commit=False)

def dequeue():
    if queue.qsize() == 0:
        return(503, f"Queue is empty")
    params = queue.get()
    queue.task_done()
    return(200, params)

def enqueue(params):
    if queue.qsize() >= max_queue_size:
       return(503, f"Queue is full")
    queue.put(params)
    queue.task_done()
    return(200, "Enqueued successfully")
0

There are 0 answers