How can I make redis.asyncio ConnectionPool work in frameworks that create multiple event loops (such as django ninja or django channels)?

107 views Asked by At

Background

I am trying to use redis.asyncio (previously aioredis) in order to manage websocket-sent notifications in my python backend. The notifications are sent by the group_send method on the channel layer provided by Django Channels. Apart from using redis as channel layer (which works fine), I am also using it as a "notification queue", so that, before sending a notification through group_send, I check if the user is online, and, if not, I queue the notification by rpush a json string. Later, when the user connects, I send the json strings from the Channels consumer.

Problem

The problem seems to be that, when I instantiate the redis connection pool, I cannot later use connections created from it in an event loop different from the one it was created in (which seems to be essential in frameworks like Channels or Django Ninja). If i instantiate it at module-level, I get "Event Loop is closed" errors by asyncio when accessing it in my Channels consumers or in my Django Ninja async views.

I tried instantiating a new connection pool on every django ninja request and on the connect() method of a Django Channels consumer. It seems to work, but isn't that very inefficient? Also, i find it very ugly to pass around the connection pool from function to function so that it can be used by a low-level function. Adding it to the self of the Channels Consumer also doesn't seem like a good idea since everything that uses redis would have to become an instance method (and then my REST api would need to duplicate code). I also don't like creating a class to maintain the connection pool as an instance variable self.connection_pool. Is the only solution to instantiate it frequently? Or is there a way to use one async connection pool for my whole backend?

0

There are 0 answers