I have an application (Liferay + Tomcat + PostgreSQL). Tomcat also has configuration for Atomikos as transaction manager:
<Resource
name="jdbc/liferay"
auth="Container"
type="com.atomikos.jdbc.AtomikosDataSourceBean"
factory="com.atomikos.tomcat.EnhancedTomcatAtomikosBeanFactory"
uniqueResourceName="jdbc/liferay"
maxPoolSize="500"
minPoolSize="10"
borrowConnectionTimeout="30"
maxIdleTime="60"
reapTimeout="0"
maintenanceInterval="60"
testQuery="SELECT 1"
xaDataSourceClassName="org.postgresql.xa.PGXADataSource"
xaProperties.serverName="${host}"
xaProperties.portNumber="${port}"
xaProperties.user="${user}"
xaProperties.databaseName="${database}"
xaProperties.password="${password}"
/>
<Resource
name="jdbc/service"
auth="Container"
type="com.atomikos.jdbc.AtomikosDataSourceBean"
factory="com.atomikos.tomcat.EnhancedTomcatAtomikosBeanFactory"
uniqueResourceName="jdbc/service"
maxPoolSize="100"
minPoolSize="10"
borrowConnectionTimeout="30"
maxIdleTime="60"
reapTimeout="0"
maintenanceInterval="60"
testQuery="SELECT 1"
xaDataSourceClassName="org.postgresql.xa.PGXADataSource"
xaProperties.serverName="${host}"
xaProperties.portNumber="${port}"
xaProperties.databaseName="service"
xaProperties.user="${user}"
xaProperties.password="${password}"
/>
At random moments (it can be few times in hour) application gets stuck and doesn't process any responses.
I got threads dump and every time when application got stuck, there is a spike of threads. You can check it on screenshots here and here. Most of this threads are in BLOCKED state e.g.:
"http-apr-443-exec-163" - Thread t@17821
java.lang.Thread.State: BLOCKED
at com.atomikos.datasource.pool.ConnectionPool.borrowConnection(ConnectionPool.java:127)
- waiting to lock <8e7fbf9> (a com.atomikos.datasource.pool.ConnectionPool) owned by "http-apr-443-exec-10" t@223
But this thread (http-apr-443-exec-10) is in RUNNABLE state:
"http-apr-443-exec-10" - Thread t@223
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.postgresql.core.VisibleBufferedInputStream.readMore(VisibleBufferedInputStream.java:161)
at org.postgresql.core.VisibleBufferedInputStream.ensureBytes(VisibleBufferedInputStream.java:128)
at org.postgresql.core.VisibleBufferedInputStream.ensureBytes(VisibleBufferedInputStream.java:113)
at org.postgresql.core.VisibleBufferedInputStream.read(VisibleBufferedInputStream.java:73)
at org.postgresql.core.PGStream.receiveChar(PGStream.java:443)
at org.postgresql.core.v3.ConnectionFactoryImpl.enableSSL(ConnectionFactoryImpl.java:505)
at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:149)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:213)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:51)
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:223)
at org.postgresql.Driver.makeConnection(Driver.java:465)
at org.postgresql.Driver.connect(Driver.java:264)
Any ideas what can it be? Deadlock or miss configuration of Atomikos?