Module Extension C/CPI Question

[Jeremy Moles]

Just a quick question before I waste time doing something that will
immediately be disregarded as unacceptable code... :)

When using the C API and writing extension modules, how do you normally
pass a structure up into the python module? For instance, if I have a
structure:

typedef struct Foo {
int x;
int y;
int z;
char xyz[100];
} Foo;

Is there an "accepted" way of propagating this upstream? I was thinking
one of these two:

1. Returning a enormous tuple of the struct's members using
Py_BuildValue("(iiis)", f.x, f.y, f.z, f.xyz) which would later be
converted to a similar object in Python for clients of my module to use.

2. Creating a PyObject and calling PyObject_SetAttrString(O, "x",
Py_BuildValue("i", f.x)) for each member, finally returning a generic
PyObject using the "O" flag in BuildValue.

Option 1 I "know" is possible; 2 I haven't tried. The object is
substantially larger and more complicated than what I used here as an
example, so Option 2 would require a bit more work. However, I'd be more
than glad to do it if the community sees it as being a cleaner solution.

Or, perhaps, there's even a better way? I'm getting more and more
experienced with the Python/C API, so I don't need a walk-through or
anything. :) Just an experienced recommendation...

[Gregory Bond]

Jeremy Moles wrote:

Or, perhaps, there's even a better way? I'm getting more and more
experienced with the Python/C API, so I don't need a walk-through or
anything. :) Just an experienced recommendation...

Better by far to create a new python type in C that has your struct
inside it and use PyMemberDefs to make those members automagically
available to Python. This has the monster advantage that members are
only converted from C objects to Python objects if they are referenced -
which is likely to have serious performance advantages if (as seems
likely with really large data objects) most of the members of most of
the objects aren't ever referenced.

See the "Embedding & Extending" document, Chapter 2.

[Martin v. Löwis]

Jeremy Moles wrote:

typedef struct Foo {
int x;
int y;
int z;
char xyz[100];
} Foo;

Is there an "accepted" way of propagating this upstream? I was thinking
one of these two:

It really depends on the struct. Would a C programmer automatically
be able to list all members, in their natural order? If they are
'x','y','z', I would say 'yes': it's obviously a point in a cartesian
coordinate system. If the members are readily known, and if you
would not use one without the others, a tuple is the right structure.

If, on the other hand, different applications would access different
fields, and if there are many fields so nobody could memorize them,
you would be better off with named access. A dictionary would work,
but people would prefer attribute names.

A typical example is the result of os.stat(), which is a type
named stat_result these days. It used to be a tuple, but that was
very inconvenient. For backwards compatibility, it had to remain
tuple-like, so we now have the StructSeq mechanism. This might
be the easiest way to get a struct-like value back to Python.

Regards,
Martin

[Martin v. Löwis]

Jeremy Moles wrote:

typedef struct Foo {
int x;
int y;
int z;
char xyz[100];
} Foo;

Is there an "accepted" way of propagating this upstream? I was thinking
one of these two:

It really depends on the struct. Would a C programmer automatically
be able to list all members, in their natural order? If they are
'x','y','z', I would say 'yes': it's obviously a point in a cartesian
coordinate system. If the members are readily known, and if you
would not use one without the others, a tuple is the right structure.

If, on the other hand, different applications would access different
fields, and if there are many fields so nobody could memorize them,
you would be better off with named access. A dictionary would work,
but people would prefer attribute names.

A typical example is the result of os.stat(), which is a type
named stat_result these days. It used to be a tuple, but that was
very inconvenient. For backwards compatibility, it had to remain
tuple-like, so we now have the StructSeq mechanism. This might
be the easiest way to get a struct-like value back to Python.

Regards,
Martin