Access a boost::interprocess shared memory block from python with ctypes

Question

I have a structure in a C++ program, running on windows, that I want to access through shared memory in Python using ctypes. For instance:

#define MAX_ENTITIES 30

struct State
{
   double x;
   double y;
   double z;
};

struct Stat
{
   unsigned int numAvailable;
   unsigned int numUsed;
};

struct TransferData
{
   double exLg;
   float other;
   unsigned int more;
   int more2;
   unsigned char next;
   bool statusReady;

   Stat status;
   State entities[MAX_ENTITIES];
};

As:

import ctypes

MAX_ENTITIES = 30


class State(ctypes.Structure):
    _fields_ = [
        ('x', ctypes.c_double),
        ('y', ctypes.c_double),
        ('z', ctypes.c_double)
    ]


class Stat(ctypes.Structure):
    _fields_ = [
        ('numAvailable', ctypes.c_uint),
        ('numUsed', ctypes.c_uint)
    ]


class TransferData(ctypes.Structure):
    _fields_ = [
        ('exLg', ctypes.c_double),
        ('other', ctypes.c_float),
        ('more', ctypes.c_uint),
        ('more2', ctypes.c_int),
        ('next', ctypes.c_ubyte),
        ('statusReady', ctypes.c_bool),
        ('status', Stat),
        ('entities', State * MAX_ENTITIES)
    ]

I was hoping that:

shmem = mmap.mmap(-1, ctypes.sizeof(TransferData.TransferData), 
          "TransferDataSHMEM")
data = TransferData.from_buffer(shmem)

Would make data the shared memory mirror of what is represented in the C++ side, but it is all zero.

Answer 1

The trick I found was actually on the boost::interprocess side. Instead of making a normal shared memory region:

  shared_memory_object shmem(open_or_create, "CppTransferDataSHMEM", read_write);
  shmem.truncate(sizeof(TransferData));
  mapped_region region(shmem, read_write);
  TransferData* data = reinterpret_cast<TransferData*>(region.get_address());

Python on windows (with the -1 file descriptor) requires that the memory be mapped to the pagefile. Boost enables this, but with an alternative windows_shared_memory instead of the default shared_memory_object.

The working code is as follows:

  windows_shared_memory shmem(create_only, "TransferDataSHMEM", 
     read_write, sizeof(TransferData));
  mapped_region region(shmem, read_write);
  std::memset(region.get_address(), 0, sizeof(TransferData));

  TransferData* data = reinterpret_cast<TransferData*>(region.get_address());

I've created a github repo with my complete example solution.