Connectivity lost between Hikari and PGBouncer

74 views Asked by At

I have a spring-boot application running on three Azure VMs using hikary as application connection pool and PG bouncer as PostgreSql Flexible Server connection pool. The application is made of dozens of micro services running in each of the three VMs.

I encountered the following exceptions during a long running load test where only some micro services lost connectivity with the DB in all three nodes :

  1. org.springframework.orm.jpa.JpaSystemException: Unable to commit against JDBC Connection; nested exception is org.hibernate.TransactionException: Unable to commit against JDBC Connection
] with root cause org.postgresql.util.PSQLException: FATAL: server conn crashed?

  2. java.sql.SQLTransientConnectionException / HikariPool-***** - Connection is not available, request timed out after *****.

These are the configurations for hikary pool for all the micro services: hikari: maximum-pool-size: 50 minimum-idle: 3 leak-detection-threshold: 2000 idle-timeout: 600000 connection-timeout: 30000 max-lifetime: 1800000

These are the PG bouncer configurations in the Azure PostgreSql Flexible Server:

PG Bouncer configuration

We didn’t reach any limit configured in hikary or pgbouncer and this was the behavior of some connection parameters taken from Azure db console:

Azure metrics

Where the blu line are the pgbouncer active client connections, the red line are the active connections on the server and the cyan line represents the pgbouncer active server connections.

Did anyone have the same issue?

I would like to try to configure the socketTimeout and statementTimeout in the hikari settings and see if this can help prevent any possible network glitches. What do you think?

0

There are 0 answers