I have a Greenplum database cluster that sometimes failed to execute SQLs because malloc() returned NULL or fork() failed due to insufficient memory (I followed the official guideline to disable overcommit). And I am curious the condition of such thing happening. Is this happens when available returned by free equals 0? Or does it has something to do with SReclaimable in /proc/meminfo? Will enabling huge page (sudo sysctl -w vm.nr_hugepages=3170) make things different?
P.S. I know that malloc() returns NULL when system is "Out-of-Memory" as already described in https://stackoverflow.com/a/59850784/8711684, but what I want to know is the exact condition that Linux kernel deem the system "Out-of-Memory" so that it will return NULL to a malloc() call with overcommit already disabled.
Any information will be appreciated!