Embedding/Extending Python in/with C++: non-static members?

[dmoore]

Hi Folks:

I have a question about the use of static members in Python/C
extensions. Take the simple example from the "Extending and Embedding
the Python Interpreter" docs:

A simple module method:

static PyObject *
spam_system(PyObject *self, PyObject *args)
{
...
}

listed in a method table:

static PyMethodDef SpamMethods[] = {
...
{"system", spam_system, METH_VARARGS,
"Execute a shell command."},
...
{NULL, NULL, 0, NULL} /* Sentinel */
};

that is registered with:

PyMODINIT_FUNC
initspam(void)
{
(void) Py_InitModule("spam", SpamMethods);
}

I have an application that embed multiple interpreters. I want to be
able to register methods for each interpreter that access C++ state
data relevant to that particular interpreter. In other words I want to
be able to register non-static class members in a class like the
following:

class PythonSpamModuleInstance
{
PythonSpamModuleInstance()
{
SpamMethods[0]={"system", spam_system, METH_VARARGS,"Execute
a shell command."};
SpamMethods[1]={NULL, NULL, 0, NULL};
PyObject *spammodule=Py_InitModule("spam", SpamMethods); //
Needs an INCREF call?
}
~PythonSpamModuleInstance()
{
PyObject_Del("spam", SpamMethods);
}
PyObject *spam_system(PyObject *self, PyObject *args)
{
// accesses non-static data in this class
}
PyMethodDef SpamMethods[2];
PyObject *spammodule;
// Other data specific to this instance of the module
};

The idea is that each time my app launches an embedded Interpreter it
will create an accompanying instance of PythonSpamModuleInstance that
allows the Interpreter to call the registered non-static C++ member
function spam_system. Calls to that member function will also affect
state information.

So the questions: will this work? In particular, will the use of non-
static member functions cause problems? Is there a better way of
creating extension modules for multiple embedded interpreters? If I am
forced to use static methods, how do i figure out which interpreter is
calling them?

[Nick Craig-Wood]

dmoore <da**********@gmail.comwrote:

I've obviously spent too much time with dynamic languages. The problem
with what I'm trying to do is obvious: In C++ you simply can't pass
pointers to call a particular instance of a C++ class method so there
is no way to initialize the PyMethodDef with class methods instances.

It is possible in theory. You can do what python does when it makes a
bound method and build a custom function at run time with the instance
and method built in.

How you build that function is tricky - you've basically got to poke
object code onto the heap which implements the method call to that
particular instance.

Looking at this page might give you some ideas

http://gcc.gnu.org/onlinedocs/gccint/Trampolines.html

This probably isn't a good approach in reality though as it is very
architecture / compiler dependent!

--
Nick Craig-Wood <ni**@craig-wood.com-- http://www.craig-wood.com/nick

[AnonMail2005]

On Jul 16, 9:45 am, dmoore <[e-mail]:

Hi Folks:

I have a question about the use of static members in Python/C
extensions. Take the simple example from the "Extending and Embedding
the Python Interpreter" docs:

A simple module method:

static PyObject *
spam_system(PyObject *self, PyObject *args)
{
...

}

listed in a method table:

static PyMethodDef SpamMethods[] = {
...
{"system", spam_system, METH_VARARGS,
"Execute a shell command."},
...
{NULL, NULL, 0, NULL} /* Sentinel */

};

that is registered with:

PyMODINIT_FUNC
initspam(void)
{
(void) Py_InitModule("spam", SpamMethods);

}

I have an application that embed multiple interpreters. I want to be
able to register methods for each interpreter that access C++ state
data relevant to that particular interpreter. In other words I want to
be able to register non-static class members in a class like the
following:

class PythonSpamModuleInstance
{
PythonSpamModuleInstance()
{
SpamMethods[0]={"system", spam_system, METH_VARARGS,"Execute
a shell command."};
SpamMethods[1]={NULL, NULL, 0, NULL};
PyObject *spammodule=Py_InitModule("spam", SpamMethods); //
Needs an INCREF call?
}
~PythonSpamModuleInstance()
{
PyObject_Del("spam", SpamMethods);
}
PyObject *spam_system(PyObject *self, PyObject *args)
{
// accesses non-static data in this class
}
PyMethodDef SpamMethods[2];
PyObject *spammodule;
// Other data specific to this instance of the module

};

The idea is that each time my app launches an embedded Interpreter it
will create an accompanying instance of PythonSpamModuleInstance that
allows the Interpreter to call the registered non-static C++ member
function spam_system. Calls to that member function will also affect
state information.

So the questions: will this work? In particular, will the use of non-
static member functions cause problems? Is there a better way of
creating extension modules for multiple embedded interpreters? If I am
forced to use static methods, how do i figure out which interpreter is
calling them?

test posting.

[dmoore]

thanks for the responses Nick and "AnonMail"

I'm doing a similar thing, and I would imagine others are also.

1. In a python file, set up wrapper functions that the python user
actually uses (e.g FunctionA). Each function corresponds to a
particular C/C++ extension function that you are exposing (e.g.
CFunctionA).

....

4. Now when you enter you C/C++ function you can find your object by
looking
it up in some predefined table using the id. What we actually do is
not use
an integer but instead use a void pointer which is cast appropriately
once
inside C/C++. This avoids the lookup.

step 4 is smart :)

It's good to know that there is at least one data point of success
using the approach. I won't give up just yet.