Need help with C extension module

[chris]

This is my first attempt at undertaking a C extension module. I want
to wrap an existing C library so I can call the functions from Python.
There are only two functions I'm interested in calling. I did mess
with Pyrex a bit and Swig, to no avail, so I turned to doing it by
hand. Using the example in Programming Python, I did get the easier of
the two functions working--only takes a string parameter. I'm stuck
now on the other function and not sure how to wrap it, because it
involves some structs. Here's a simplified version of the C:

struct In
{
int x;
char* s;
... (only primitive data types)
};

struct Out
{
int y;
char* s;
... (only primitive data types)
};

Out* func(In* x, Out* y);

So the function takes pointers to the two structs, and fills out the
output struct and also returns the pointer to it. I would envision the
Python looking like

in = In()
in.y = 1
in.s = "abc"
....

out = func(in)

maybe? Just no idea how to deal with the structs in the C extension
module code.

Any tips appreciated.

Thanks,
Chris

[Robert Kern]

chris wrote:

This is my first attempt at undertaking a C extension module. I want
to wrap an existing C library so I can call the functions from Python.
There are only two functions I'm interested in calling. I did mess
with Pyrex a bit and Swig, to no avail, so I turned to doing it by
hand. Using the example in Programming Python, I did get the easier of
the two functions working--only takes a string parameter. I'm stuck
now on the other function and not sure how to wrap it, because it
involves some structs. Here's a simplified version of the C:

struct In
{
int x;
char* s;
... (only primitive data types)
};

struct Out
{
int y;
char* s;
... (only primitive data types)
};

Out* func(In* x, Out* y);

So the function takes pointers to the two structs, and fills out the
output struct and also returns the pointer to it. I would envision the
Python looking like

in = In()
in.y = 1
in.s = "abc"
...

out = func(in)

maybe? Just no idea how to deal with the structs in the C extension
module code.

It's *not* a straightforward task. You have to define an extension type. See

http://docs.python.org/ext/defining-new-types.html

However, Pyrex really does make defining extension types much, much
easier. I highly suggest trying to debug your Pyrex code before learning
all of the boilerplate necessary to create extension types in pure C.

--
Robert Kern
rk***@ucsd.edu

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

[chris]

Any tips on what the pyrex should look like for my example?

[Robert Kern]

chris wrote:

Any tips on what the pyrex should look like for my example?

# Untested and off the top of my head; please read the Pyrex
# documentation and correct my mistakes before using any of this!
# Notably, I'm pretty sure the __init__ and __dealloc__ bits are
# slightly wrong.

cdef extern from "Python.h":
void *PyMem_Malloc(unsigned int size)
void PyMem_Free(void *buf)

ctypedef struct In:
int x
char *s
# ...

ctypedef struct Out:
int y
char *s
# ...

cdef Out *func(Int *x, Out *y)

cdef class Output:
cdef Out *output

def __init__(self):
self.output = <Out*>PyMem_Malloc(sizeof(Out))

def __dealloc__(self):
PyMem_Free(self.output)

property y:
def __get__(self):
return self.output.y

# ...

cdef class Input:
cdef In *input

def __init__(self):
self.input = <In*>PyMem_Malloc(sizeof(In))

def __dealloc__(self):
PyMem_Free(self.input)

property x:
def __get__(self):
return self.input.x

def __set__(self, value):
self.input.x = value

# ...

def func(self):
output = Output()
func(self.input, output.output)
return output

--
Robert Kern
rk***@ucsd.edu

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

[chris]

Many thanks Robert. That will be a good starting point.

-Chris

http://auslunch.com/
http://fetidcascade.com/
http://strombergers.com/python/

[chris]

Ok, I found further examples on the Internet and got something working
(it seems), but I have a question about the memory management. The
example I found did not include any of the PyMem_... functions.

Here's roughly what I have working:

cdef extern from "my.h":
cdef struct inputs:
char *x

cdef struct outputs:
int y

outputs *func(inputs *x, outputs *y)
int init(char* fname)

class Analyzer:
def __init__(self, fname):
init(fname)

# inp is my python "Inputs" object.
def myfunc(self, inp):
cdef inputs* i
i.x= inp.x

cdef outputs* o
o = func(i, o)
return o.y

class Inputs:
def __init__(self):
self.x = ""

So there is no explicit memory management going on there as in Robert's
example. Is this ok?

Thanks,
Chris

[Ken Seehart]

chris wrote:

This is my first attempt at undertaking a C extension module. I want
to wrap an existing C library so I can call the functions from Python.
There are only two functions I'm interested in calling. I did mess
with Pyrex a bit and Swig, to no avail, so I turned to doing it by
hand. Using the example in Programming Python, I did get the easier of
the two functions working--only takes a string parameter. I'm stuck
now on the other function and not sure how to wrap it, because it
involves some structs. Here's a simplified version of the C:

struct In
{
int x;
char* s;
... (only primitive data types)
};

struct Out
{
int y;
char* s;
... (only primitive data types)
};

Out* func(In* x, Out* y);

So the function takes pointers to the two structs, and fills out the
output struct and also returns the pointer to it. I would envision the
Python looking like

in = In()
in.y = 1
in.s = "abc"
...

out = func(in)

maybe? Just no idea how to deal with the structs in the C extension
module code.

Any tips appreciated.

Thanks,
Chris

Since others have responded about Pyrex, I'll just add my two cents,
which may or not apply to you.

The idea of filling in a struct, passing it to a function, then
returning a struct, is a typical c-ism. If the only purpose of the
structs is to pass data to the function and receive the result, read on.
On the other hand, if the structures represent "things" in the
object-oriented sense, just ignore what I am saying.

I would wrap the function such that the python code simply takes a set
of parameters and returns a tuple. This is much more pythonic, and much
easier to code.

y,s,... = func(x,s,...)

My untested pseudocode would look something like this:

static PyObject *map_sweep(map__object *self, PyObject *args)
{
In input;
Out output;

if (PyArg_ParseTuple(args, "is...", &(In.x), &(In.s), ...))
{
return NULL;
}

func(&In, &Out);

return Py_BuildValue("is...", Out.y, Out.s, ...);
}