473,543 Members | 2,172 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

segfault in extension module


I've written a function in C to perform protein sequence alignment. This
works fine in a standalone C program. I've added the necessary packaging
to use it in Python; it returns three strings and an integer. However, as
soon as the function is complete, I get a segfault and the interpreter
dies.

If I run Python interactively, just calling the function causes a
segfault. If I'm running a script, I can actually print out the return
values (which are what I'd expect - so something's working) but as soon as
the script is done I get the segfault again. I can even call the function
twice, with different arguments - and it works both times. So it appears
that the problem is with tying up loose ends.

How do I determine what is going wrong? I do not get any problem like
this in the C version. I am not using free() anywhere - I will eventually
need to fix this, but I cannot find any place where I might be accessing
unavailable memory. (Adding in free() does not make any difference in the
module, for what it's worth, but I've had some issues with the C program
so I've left it out.)

[I've also tried using PyMem_Malloc instead, throughout the C code.
Doesn't help.]

thanks,
Nat
(please reply directly!)
Jul 18 '05 #1
12 3139
Nathaniel Echols <ec****@OCF.Ber keley.EDU> writes:
How do I determine what is going wrong?


I recommend to run your code (or the interactive python) in a debugger.
For example, with gdb, you'd get

gdb /usr/bin/python
(gdb) run
import foo

[program crashes]
(gdb) bt

The latter command will give a backtrace, which should tell you where
it crashes. If you don't get enough detail, make sure you compile your
module with debugging information. If you are on Windows, make then
sure that Python is compiled for debugging as well.

Regards,
Martin
Jul 18 '05 #2
Hello Nat,
I've written a function in C to perform protein sequence alignment. This
works fine in a standalone C program. I've added the necessary packaging
to use it in Python; Which one? Are you using the C API?
it returns three strings and an integer. However, as
soon as the function is complete, I get a segfault and the interpreter
dies.

Looks like refcount problems, check out
http://www.python.org/doc/current/ext/refcounts.html .

IMO just avoid all this stuff and use SWIG/Boost.Python/Pyrex.

HTH.
Miki
Jul 18 '05 #3
> > I've written a function in C to perform protein sequence alignment. This
works fine in a standalone C program. I've added the necessary packaging
to use it in Python; Which one? Are you using the C API?


Yup - I'm reading right out of the manual.
Looks like refcount problems, check out
http://www.python.org/doc/current/ext/refcounts.html .
I read this before and couldn't figure out what it meant. This does seem
like it would relate, but I can't figure out what I'm doing incorrectly.
I just have one function which calls a pure C function and returns a
tuple of strings from it. I'm guessing I need to add a Py_INCREF()
somewhere but so far this just makes it segfault sooner. (I'm
not sure what argument to use for Py_INCREF(), either.)

I've looked at several other pages, and they all seem to involve setups
more complicated than what I'm doing. I'm already using Py_BuildValue()
to generate the returned tuple, and my understanding is that this should
avoid major problems. . .
IMO just avoid all this stuff and use SWIG/Boost.Python/Pyrex.


I'll look at these, but I only have a tiny little bit of code I need to do
this with - I coded it from scratch with the intention of using it this
way, and could have written it in Python if I didn't care about speed.
Would I really benefit from using one of the other methods? The goal here
is explicitly to put only the very time-dependent code in C; everything
else stays in Python.

thanks,
Nat
Jul 18 '05 #4
> The latter command will give a backtrace, which should tell you where it
crashes. If you don't get enough detail, make sure you compile your
module with debugging information. If you are on Windows, make then sure
that Python is compiled for debugging as well. Regards, Martin


Okay:

#0 0x420744fe in _int_free () from /lib/tls/libc.so.6
#1 0x420734d6 in free () from /lib/tls/libc.so.6
#2 0x0809dc0d in _PyObject_GC_De l ()
#3 0x080ce86f in PyDict_Next ()
#4 0x080d191d in _PyModule_Clear ()

I guess this makes sense, but I'm still not sure how to fix it. . .
Jul 18 '05 #5
Nathaniel Echols <ec****@OCF.Ber keley.EDU> writes:
[...]
IMO just avoid all this stuff and use SWIG/Boost.Python/Pyrex.
I'll look at these, but I only have a tiny little bit of code I need to do

[...] Would I really benefit from using one of the other methods?
Why not take advantage of it? You've already discovered how
hand-writing extensions can be painful. SWIG is probably best for
you. Ignore all the fancy SWIG features, just ask it to wrap the
function, it's very easy.

The goal here
is explicitly to put only the very time-dependent code in C; everything
else stays in Python.


SWIG should be able to do that. There's probably some overhead above
a hand-written wrapper, but it's so trivial to use SWIG in this simple
way that you probably shouldn't begin to worry about that -- it's
unlikely to be a problem.
John
Jul 18 '05 #6
Nathaniel Echols <ec****@OCF.Ber keley.EDU> writes:
Looks like refcount problems, check out
http://www.python.org/doc/current/ext/refcounts.html .
I read this before and couldn't figure out what it meant.


Then I am pretty sure this is your problem :-)
This does seem like it would relate, but I can't figure out what I'm
doing incorrectly. I just have one function which calls a pure C
function and returns a tuple of strings from it. I'm guessing I
need to add a Py_INCREF() somewhere but so far this just makes it
segfault sooner. (I'm not sure what argument to use for
Py_INCREF(), either.)

I've looked at several other pages, and they all seem to involve setups
more complicated than what I'm doing. I'm already using Py_BuildValue()
to generate the returned tuple, and my understanding is that this should
avoid major problems. . .


Post some code.

Cheers,
mwh

--
If Unicode is a horde of zombies with flaming dung sticks,
the hideous intricacies of JIS, Chinese Big-5, Chinese
Traditional, KOI-8, et cetera are at least an army of ogres
with salt and flensing knives. -- Eric S. Raymond, python-dev
Jul 18 '05 #7
Nathaniel Echols <ec****@OCF.Ber keley.EDU> writes:
Okay:

#0 0x420744fe in _int_free () from /lib/tls/libc.so.6
#1 0x420734d6 in free () from /lib/tls/libc.so.6
#2 0x0809dc0d in _PyObject_GC_De l ()
#3 0x080ce86f in PyDict_Next ()
#4 0x080d191d in _PyModule_Clear ()

I guess this makes sense, but I'm still not sure how to fix it. . .


Ah, tls/libc.so.6. I think you lose, being confronted with a buggy C
library. Try making it not use /lib/tls.

If that does not change the behaviour, you probably have a
ref-counting bug somewhere. Try building a debugging version of
Python.

Regards,
Martin
Jul 18 '05 #8
> Post some code.

may god have mercy on my soul:

#include <Python.h>
#include "nw.h"
static PyObject *nw_align (PyObject *self, PyObject *args) {
char *seq1, *seq2, *mfile;
char *out1, *out2, *match;
int penalty, status, score;
PyObject *results;

if (! PyArg_ParseTupl e(args, "sssi", &seq1, &seq2, &mfile, &penalty)) {
return NULL;
}

status = nw(seq1, seq2, mfile, penalty, &out1, &out2, &match, &score);

if (status == -1) {
PyErr_NoMemory( );
return NULL;
}

results = Py_BuildValue(" (sssi)", out1, out2, match, score);
return results;
}

static PyMethodDef nwMethods[] = {
{"align", nw_align, METH_VARARGS,
"Perform Needleman-Wunsch alignment of two protein sequences."},
{NULL, NULL, 0, NULL} /* This is required! No idea why. */
};

void initnw (void) {
(void) Py_InitModule(" nw", nwMethods);
}

This is just the wrapper; the function that it calls is defined such:
int nw (char *seq1, char *seq2, char *matrixfile, int penalty,
char **_out1, char **_out2, char **_match, int *_score);
(I can supply this too, but I already know this works fine in a standalone
C program.)

In Python, I simply do this:
(out1, out2, match, score) = nw.align(seq1, seq2, "BLOSUM62", 0)

This is it; should be simple to fix, no?
Jul 18 '05 #9
Nat Echols <ec****@uclink. berkeley.edu> writes:
Post some code.


may god have mercy on my soul:


Well, I can't see anything flagrantly wrong with that.

Does a debugger provide any hints?

Cheers,
mwh

--
[3] Modem speeds being what they are, large .avi files were
generally downloaded to the shell server instead[4].
[4] Where they were usually found by the technical staff, and
burned to CD. -- Carlfish, asr
Jul 18 '05 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
1365
by: Robert Frunzke | last post by:
Hello, I'm developing a php extension as a communication layer for a proprietary database. Now, when I run two different versions of this extension on the same webserver (Apache 3, PHP 4.3.3, extension loaded via dl() ), I will get strange segmentation faults - that occur somewhere in PHP (not in the extension itself).
0
1791
by: David Eger | last post by:
I'm trying to make a very simple extension which will return a new copy of a C++ string object to python, and I'm segfaulting in the process. I'm using Python 2.2.2, SWIG 1.3.17 and g++ 3.3. Am I getting something very basic wrong? See code below. -David /* file: dups.cxx */ #include "dups.h"
6
3003
by: Juho Saarikko | last post by:
The program attached to this message makes the Python interpreter segfault randomly. I have tried both Python 2.2 which came with Debian Stable, and self-compiled Python 2.3.3 (newest I could find on www.python.org, compiled with default options (./configure && make). I'm using the pyPgSQL plugin to connect to a PostGreSQL database, and have...
8
2751
by: Torsten Mohr | last post by:
Hi, i write an extension module in C at the moment. This module does some work on some own data types that consist of some values. The functions that can change the data are written in C. The question came up if this is by itself thread safe, if some two or more threads try to change these data types, are the C functions by themselves...
3
3359
by: Travis Berg | last post by:
I'm running into a problem when trying to perform a callback to a Python function from a C extension. Specifically, the callback is being made by a pthread that seems to cause the problem. If I call the callback from the parent process, it works fine. The PyObject is static, and holds the same value in both Parent and thread, so I'm at a...
4
1872
by: klappnase | last post by:
Hello, I use the tktreectrl Tk extension (http://tktreectrl.sourceforge.net) through the python wrapper module (http://klappnase.zexxo.net/TkinterTreectrl/index.html). With python-2.5 it seems that each time some text is inserted into the treectrl widget a segfault occurs (on linux, on windows2000 it seemed to work). Some people sent me...
14
4940
by: Donn Ingle | last post by:
Yo, An app of mine relies on PIL. When PIL hits a certain problem font (for unknown reasons as of now) it tends to segfault and no amount of try/except will keep my wxPython app alive. My first thought is to start the app from a bash script that will check the return value of my wxPython app and could then launch a new app to help the user...
3
2684
by: Mitko Haralanov | last post by:
I have a Python module that I have written using the C API and I am having a problem accessing a dictionary from that module. Here is what I have done: 1. In my init function I call module = Py_InitModule ("name", methods); 2. then I get the module's __dict__ structure: dict = PyModule_GetDict (module); 3. Later, I insert a dictionary...
18
1498
by: Prasad | last post by:
Hi folks, I am trying to debug the following program. Debugging the core file revealed segfault at possibleFlows=0 when values of states=36, labels=40. It works fine for lesser values. What could be the reason? Integer overflow or out of memory? Am i missing something obvious? I am using ubuntu and GNU g++.
0
7590
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. ...
0
7735
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that...
1
7347
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For...
0
7688
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the...
1
5271
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes...
0
4895
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...
1
1817
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
1
968
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
0
636
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.