How to clean apache ignite free ram?

Question

I am facing a recurring issue with Apache Ignite where the application crashes approximately once a month due to continuous depletion of free RAM. I am utilizing Ignite cache for storing queryable entities in my application. Even after destroying the cache, the free RAM continues to decrease until it reaches about 3%-%1 just before the crash. Restarting Ignite brings the free RAM back to 100%.

In my Ignite configuration, I have set the eviction policy to "RANDOM_2_LRU."

<property name="dataStorageConfiguration">
            <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
                <property name="defaultDataRegionConfiguration">
                    <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
                        <property name="name" value="Default_Region"/>
                        <property name="initialSize" value="#{256 * 1024 * 1024}"/>
                        <property name="maxSize" value="#{80L * 1024 * 1024 * 1024}"/>
                        <property name="pageEvictionMode" value="RANDOM_2_LRU"/>
            <property name="swapPath" value="/swap"/>
                    </bean>
                </property>
            </bean>
        </property>

before crush after restart

I would like to seek guidance on effectively managing and handling the free RAM to prevent these crashes. Is there a way to automatically clean the RAM or configure specific settings to mitigate this issue?

Any specific configurations or settings that can help address this RAM depletion issue? Any insights or suggestions on managing memory effectively with Apache Ignite?

Answer 1

Based on your comment, I think you're overcommitting your container. Your Ignite configuration looks fine, and it will not use more than 80Gb of off-heap memory.

However, Ignite doesn't only use off-heap memory. You also need to take into consideration the Java heap. Java itself takes up some memory, as does the operating system.

In short, if your container is configured to allow 80Gb of memory, your Ignite data region must be less than that. If your Java heap is 6Gb (you don't say), then I would allow around 10Gb total, meaning that your off-heap size should be around 70Gb.

Confirmed from your logs:

[WARNING][main][IgniteKernal] Nodes started on local machine require more than 80% of physical RAM what can lead to significant slowdown due to swapping (please decrease JVM heap size, data region size or checkpoint buffer size) [required=145736MB, available=128755MB]

With 128Gb pod size, I'd do something like 8Gb Java heap, 110Gb off-heap data region. More tuning information can be found in the documentation.