473,320 Members | 1,870 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

segfault when calling Python from C thread


I'm running into a problem when trying to perform a callback to a Python
function from a C extension. Specifically, the callback is being made by
a pthread that seems to cause the problem. If I call the callback from
the parent process, it works fine. The PyObject is static, and holds the
same value in both Parent and thread, so I'm at a loss as to what would be
different in the pthread from the parent that would cause a segfault on
the callback. The machine specifics are an x86 intel processor with
RedHat linux.

Here is some clips of the C callback code showing how I'm storing the
callback, then what the actual callback function is like. Any ideas?
The function being called is a simply to display the string of text, and
execution never seems to reach back to the Python code at all.

Thanks,
Travis B.
/* callback function to the Python code */
static PyObject * my_callback = NULL;

/* setting callback function */
static PyObject * my_set_callback(PyObject *dummy, PyObject *args)
{
PyObject *result = NULL;
PyObject *temp;
PyObject *arglist;

if (PyArg_ParseTuple(args, "O:set_callback", &temp)) {
if (!PyCallable_Check(temp)) {
PyErr_SetString(PyExc_TypeError,
"parameter must be callable");
return NULL;
}
Py_XINCREF(temp); /* Add a reference to new callback */
Py_XDECREF(my_callback); /* Dispose of previous callback */
my_callback = temp; /* Remember new callback */
/* return "None" */
Py_INCREF(Py_None);
result = Py_None;
}
return result;
}

/* calling callback */
void callback(char * str) {
PyObject *arglist;
PyObject *result;
if(str == NULL)
return;

if(my_callback == NULL) {
printf("no callback function provided, returning...\n");
return;
}

/* Time to call the callback */
arglist = Py_BuildValue("(s)", str);
result = PyEval_CallObject(my_callback, arglist);
Py_DECREF(arglist);
if(result == NULL)
return;
Py_DECREF(result);
}
Jul 18 '05 #1
3 3338
Travis Berg wrote:

I'm running into a problem when trying to perform a callback to a
Python function from a C extension. Specifically, the callback is
being made by a pthread that seems to cause the problem. If I call
the callback from the parent process, it works fine. The PyObject is
static, and holds the same value in both Parent and thread, so I'm at
a loss as to what would be different in the pthread from the parent
that would cause a segfault on the callback. The machine specifics
are an x86 intel processor with RedHat linux.
/* calling callback */
void callback(char * str) {
PyObject *arglist;
PyObject *result;
if(str == NULL)
return;

if(my_callback == NULL) {
printf("no callback function provided, returning...\n");
return;
}

/* Time to call the callback */
arglist = Py_BuildValue("(s)", str);
result = PyEval_CallObject(my_callback, arglist);
Py_DECREF(arglist);
if(result == NULL)
return;
Py_DECREF(result);
}


Your callback function needs to hold the Python GIL (and have a vaild
threadstate) before it calls any Python C-API functions. Change the
last part of it to:

PyGILState_STATE state;

/* ... */

/* Time to call the callback */

state = PyGILState_Ensure();

arglist = Py_BuildValue("(s)", str);
result = PyEval_CallObject(my_callback, arglist);
Py_DECREF(arglist);
if(result == NULL)
return;
Py_DECREF(result);

PyGILState_Release(state);
}

Also, somewhere in your main thread you should call PyEval_InitThreads
before any of the callback threads execute. (This call is made
automatically if you are creating new threads using Python's thread
module, but if the new threads are created by some C code, you need to
call it yourself.)

---
Greg Chapman
Jul 18 '05 #2
Greg Chapman wrote:
Your callback function needs to hold the Python GIL (and have a vaild
threadstate) before it calls any Python C-API functions. Change the
last part of it to:

PyGILState_STATE state;

/* ... */

/* Time to call the callback */

state = PyGILState_Ensure();

arglist = Py_BuildValue("(s)", str);
result = PyEval_CallObject(my_callback, arglist);
Py_DECREF(arglist);
if(result == NULL)
return;
Py_DECREF(result);

PyGILState_Release(state);
}


you might wish to make sure you release the GIL even if the callback
raises an exception...

</F>

Jul 18 '05 #3
Fredrik Lundh wrote:
Greg Chapman wrote:
Your callback function needs to hold the Python GIL (and have a
vaild threadstate) before it calls any Python C-API functions.
Change the last part of it to:

PyGILState_STATE state;

/* ... */

/* Time to call the callback */

state = PyGILState_Ensure();

arglist = Py_BuildValue("(s)", str);
result = PyEval_CallObject(my_callback, arglist);
Py_DECREF(arglist);
if(result == NULL)
return;
Py_DECREF(result);

PyGILState_Release(state);
}


you might wish to make sure you release the GIL even if the callback
raises an exception...

</F>


Argh, thanks for catching that. You probably put that too politely
though (in case anyone sees this who might think that is optional): one
should absolutely make sure all code paths which call PyGILState_Ensure
have a matching call to PyGILState_Release.

---
Greg Chapman
Jul 18 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

12
by: Nathaniel Echols | last post by:
I've written a function in C to perform protein sequence alignment. This works fine in a standalone C program. I've added the necessary packaging to use it in Python; it returns three strings and...
6
by: Juho Saarikko | last post by:
The program attached to this message makes the Python interpreter segfault randomly. I have tried both Python 2.2 which came with Debian Stable, and self-compiled Python 2.3.3 (newest I could find...
6
by: Stefan Behnel | last post by:
Hi! In Python 2.4b3, the deque is causing a segfault on two different machines I tested on. With deque, my program runs fine for a while (at least some tens of seconds up to minutes) and then...
0
by: dale | last post by:
Python newbie disclaimer on I am running an app with Tkinter screen in one thread and command-line input in another thread using raw_input(). First question - is this legal, should it run...
1
by: Tony Meyer | last post by:
I have (unfortunately) a Python program that I can consistently (in a reproducible way) segfault. However, I've got somewhat used to Python's very nice habit of protecting me from segfaults and...
4
by: klappnase | last post by:
Hello, I use the tktreectrl Tk extension (http://tktreectrl.sourceforge.net) through the python wrapper module (http://klappnase.zexxo.net/TkinterTreectrl/index.html). With python-2.5 it seems...
0
by: bernhard.voigt | last post by:
Dear list, I'm using two external modules (matplotlib and pyroot). Both are using different GUI threads (internally) Unfortunately, pretty soon after I've loaded both modules and made some...
3
by: Paul Sijben | last post by:
I am running a multi-threaded python application in a dual core intel running Ubuntu. I am using python 2.5.1 that I compiled myself. At random points I am getting segmentation faults (sometimes...
14
by: Donn Ingle | last post by:
Yo, An app of mine relies on PIL. When PIL hits a certain problem font (for unknown reasons as of now) it tends to segfault and no amount of try/except will keep my wxPython app alive. My first...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
0
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.