473,549 Members | 2,627 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

segfault when calling Python from C thread


I'm running into a problem when trying to perform a callback to a Python
function from a C extension. Specifically, the callback is being made by
a pthread that seems to cause the problem. If I call the callback from
the parent process, it works fine. The PyObject is static, and holds the
same value in both Parent and thread, so I'm at a loss as to what would be
different in the pthread from the parent that would cause a segfault on
the callback. The machine specifics are an x86 intel processor with
RedHat linux.

Here is some clips of the C callback code showing how I'm storing the
callback, then what the actual callback function is like. Any ideas?
The function being called is a simply to display the string of text, and
execution never seems to reach back to the Python code at all.

Thanks,
Travis B.
/* callback function to the Python code */
static PyObject * my_callback = NULL;

/* setting callback function */
static PyObject * my_set_callback (PyObject *dummy, PyObject *args)
{
PyObject *result = NULL;
PyObject *temp;
PyObject *arglist;

if (PyArg_ParseTup le(args, "O:set_callback ", &temp)) {
if (!PyCallable_Ch eck(temp)) {
PyErr_SetString (PyExc_TypeErro r,
"parameter must be callable");
return NULL;
}
Py_XINCREF(temp ); /* Add a reference to new callback */
Py_XDECREF(my_c allback); /* Dispose of previous callback */
my_callback = temp; /* Remember new callback */
/* return "None" */
Py_INCREF(Py_No ne);
result = Py_None;
}
return result;
}

/* calling callback */
void callback(char * str) {
PyObject *arglist;
PyObject *result;
if(str == NULL)
return;

if(my_callback == NULL) {
printf("no callback function provided, returning...\n" );
return;
}

/* Time to call the callback */
arglist = Py_BuildValue(" (s)", str);
result = PyEval_CallObje ct(my_callback, arglist);
Py_DECREF(argli st);
if(result == NULL)
return;
Py_DECREF(resul t);
}
Jul 18 '05 #1
3 3360
Travis Berg wrote:

I'm running into a problem when trying to perform a callback to a
Python function from a C extension. Specifically, the callback is
being made by a pthread that seems to cause the problem. If I call
the callback from the parent process, it works fine. The PyObject is
static, and holds the same value in both Parent and thread, so I'm at
a loss as to what would be different in the pthread from the parent
that would cause a segfault on the callback. The machine specifics
are an x86 intel processor with RedHat linux.
/* calling callback */
void callback(char * str) {
PyObject *arglist;
PyObject *result;
if(str == NULL)
return;

if(my_callback == NULL) {
printf("no callback function provided, returning...\n" );
return;
}

/* Time to call the callback */
arglist = Py_BuildValue(" (s)", str);
result = PyEval_CallObje ct(my_callback, arglist);
Py_DECREF(argli st);
if(result == NULL)
return;
Py_DECREF(resul t);
}


Your callback function needs to hold the Python GIL (and have a vaild
threadstate) before it calls any Python C-API functions. Change the
last part of it to:

PyGILState_STAT E state;

/* ... */

/* Time to call the callback */

state = PyGILState_Ensu re();

arglist = Py_BuildValue(" (s)", str);
result = PyEval_CallObje ct(my_callback, arglist);
Py_DECREF(argli st);
if(result == NULL)
return;
Py_DECREF(resul t);

PyGILState_Rele ase(state);
}

Also, somewhere in your main thread you should call PyEval_InitThre ads
before any of the callback threads execute. (This call is made
automatically if you are creating new threads using Python's thread
module, but if the new threads are created by some C code, you need to
call it yourself.)

---
Greg Chapman
Jul 18 '05 #2
Greg Chapman wrote:
Your callback function needs to hold the Python GIL (and have a vaild
threadstate) before it calls any Python C-API functions. Change the
last part of it to:

PyGILState_STAT E state;

/* ... */

/* Time to call the callback */

state = PyGILState_Ensu re();

arglist = Py_BuildValue(" (s)", str);
result = PyEval_CallObje ct(my_callback, arglist);
Py_DECREF(argli st);
if(result == NULL)
return;
Py_DECREF(resul t);

PyGILState_Rele ase(state);
}


you might wish to make sure you release the GIL even if the callback
raises an exception...

</F>

Jul 18 '05 #3
Fredrik Lundh wrote:
Greg Chapman wrote:
Your callback function needs to hold the Python GIL (and have a
vaild threadstate) before it calls any Python C-API functions.
Change the last part of it to:

PyGILState_STAT E state;

/* ... */

/* Time to call the callback */

state = PyGILState_Ensu re();

arglist = Py_BuildValue(" (s)", str);
result = PyEval_CallObje ct(my_callback, arglist);
Py_DECREF(argli st);
if(result == NULL)
return;
Py_DECREF(resul t);

PyGILState_Rele ase(state);
}


you might wish to make sure you release the GIL even if the callback
raises an exception...

</F>


Argh, thanks for catching that. You probably put that too politely
though (in case anyone sees this who might think that is optional): one
should absolutely make sure all code paths which call PyGILState_Ensu re
have a matching call to PyGILState_Rele ase.

---
Greg Chapman
Jul 18 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

12
3140
by: Nathaniel Echols | last post by:
I've written a function in C to perform protein sequence alignment. This works fine in a standalone C program. I've added the necessary packaging to use it in Python; it returns three strings and an integer. However, as soon as the function is complete, I get a segfault and the interpreter dies. If I run Python interactively, just...
6
3004
by: Juho Saarikko | last post by:
The program attached to this message makes the Python interpreter segfault randomly. I have tried both Python 2.2 which came with Debian Stable, and self-compiled Python 2.3.3 (newest I could find on www.python.org, compiled with default options (./configure && make). I'm using the pyPgSQL plugin to connect to a PostGreSQL database, and have...
6
1976
by: Stefan Behnel | last post by:
Hi! In Python 2.4b3, the deque is causing a segfault on two different machines I tested on. With deque, my program runs fine for a while (at least some tens of seconds up to minutes) and then suddenly segfaults. I'm sorry I can't tell exactly when, but I'm running an application that uses a few hundred deques where elements are...
0
1812
by: dale | last post by:
Python newbie disclaimer on I am running an app with Tkinter screen in one thread and command-line input in another thread using raw_input(). First question - is this legal, should it run without issue? If not can you point me to a description of why. While updating objects on the screen I get a segfault after an indeterminate number...
1
1261
by: Tony Meyer | last post by:
I have (unfortunately) a Python program that I can consistently (in a reproducible way) segfault. However, I've got somewhat used to Python's very nice habit of protecting me from segfaults and raising exceptions instead, and am having trouble tracking down the problem. The problem that occurs looks something like this: Program received...
4
1872
by: klappnase | last post by:
Hello, I use the tktreectrl Tk extension (http://tktreectrl.sourceforge.net) through the python wrapper module (http://klappnase.zexxo.net/TkinterTreectrl/index.html). With python-2.5 it seems that each time some text is inserted into the treectrl widget a segfault occurs (on linux, on windows2000 it seemed to work). Some people sent me...
0
1057
by: bernhard.voigt | last post by:
Dear list, I'm using two external modules (matplotlib and pyroot). Both are using different GUI threads (internally) Unfortunately, pretty soon after I've loaded both modules and made some function calls (without invoking GUI commands), python dies with a segfault. A libthread_db captures the stack trace. My python version is 2.5 and I'm on...
3
1934
by: Paul Sijben | last post by:
I am running a multi-threaded python application in a dual core intel running Ubuntu. I am using python 2.5.1 that I compiled myself. At random points I am getting segmentation faults (sometimes indicating a duplicate free). Below is the backtrace of the latest segfault. I am thinking this might be an issue related to the dual core CPU so...
14
4944
by: Donn Ingle | last post by:
Yo, An app of mine relies on PIL. When PIL hits a certain problem font (for unknown reasons as of now) it tends to segfault and no amount of try/except will keep my wxPython app alive. My first thought is to start the app from a bash script that will check the return value of my wxPython app and could then launch a new app to help the user...
0
7521
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main...
0
7720
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. ...
1
7473
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For...
0
7810
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the...
0
6044
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then...
0
5088
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...
0
3501
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in...
1
1944
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
0
764
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.