I'm trying to do NUMA aware memory allocation with hwloc and get somewhat strange behavior.
My goal is to allocate blocks of memory on different NUMA nodes as i need this for a project. To verify that the correct amount of memory is allocated i have been using valgrind's memcheck tool. The tool reports always the same amount of bytes allocated no matter how many elements i want to allocate, which doesn't make sense to me. And overall the number of allocations seems to high. I do understand that hwloc needs to allocate some stuff internally to work properly but still this doesn't make sense to me.
Even if i try to allocate a bigger chunk of memory the allocated bytes reported by valgrind remain the same.
Here is the code i have been using to allocate memory on NUMA node 0.
#include <iostream>
#include <hwloc.h>
int main() {
hwloc_topology_t topo;
hwloc_topology_init(&topo);
hwloc_topology_load(topo);
auto node = hwloc_get_obj_by_type(topo, HWLOC_OBJ_NUMANODE, 0);
auto mem = hwloc_alloc_membind(topo, SIZE * sizeof(float), node->nodeset,
HWLOC_MEMBIND_BIND, HWLOC_MEMBIND_STRICT | HWLOC_MEMBIND_BYNODESET);
if (mem == NULL) {
std::cout << "Allocation failed" << std::endl;
}
hwloc_free(topo, mem, SIZE);
hwloc_topology_destroy(topo);
}
and this is the relevant part of the valgrind memcheck report: valgrind report
Really hope some can explain what is going on here.
I'm using hwloc 2.7.1 and valgrind 3.15.0
Valgrind 3.15 is quite old - can you try something more recent?
To see everything you need to use
valgrind --leak-check=full --show-reachable=yes --default-suppressions=noWith that I get (using a SIZE of 1024)
Also try using
--trace-malloc=yesto see all of the allocation and deallocation calls.