Access contents of PyBuffer from C

Question

I have created a buffer object in python like so:

f = io.open('some_file', 'rb')
byte_stream = buffer(f.read(4096))

I'm now passing byte_stream as a parameter to a C function, through SWIG. I have a typemap for converting the data which looks like this:

%typemap(in) unsigned char * byte_stream {
    PyObject *buf = $input;
    //some code to read the contents of buf
}

I have tried a few different things bug can't get to the actual content/value of my byte_stream. How do I convert or access the content of my byte_stream using the C API? There are many different methods for converting a C data to a buffer but none that I can find for going the other way around. I have tried looking at this object in gcb but neither it, or the values it points to contain my data.

(I'm using buffers because I want to avoid the overhead of converting the data to a string when reading it from the file) I'm using python 2.6 on Linux.

-- Thanks Pavel

Answer 1

I'm using buffers because I want to avoid the overhead of converting the data to a string when reading it from the file

You are not avoiding anything. The string is already built by the read() method. Calling buffer() just builds an additional buffer object pointing to that string.

As for getting at the memory pointed to by the buffer object, try PyObject_AsReadBuffer(). See also http://docs.python.org/c-api/objbuffer.html.

Answer 2

As soon as you use the read method on your file object, the data will be converted to a str object; calling the buffer method does not convert it into a stream of any kind. If you want to avoid the overhead of creating the string object, you could simply pass the file object to your C code and then use it via its C API.