how to debug when "Segmentation fault"

[Maksim Kasimov]

Hello,

my programm sometime gives "Segmentation fault" message (no matter how long the programm had run (1 day or 2 weeks). And there is nothing in log-files that can points the problem.
My question is how it possible to find out where is the problem in the code? Thanks for any help.

Python 2.2.3
FreeBSD

--
Best regards,
Maksim Kasimov
mailto: ma************@gmail.com

[Michael Ekstrand]

On Tuesday 04 October 2005 11:13, Maksim Kasimov wrote:

my programm sometime gives "Segmentation fault" message (no matter
how long the programm had run (1 day or 2 weeks). And there is
nothing in log-files that can points the problem. My question is how
it possible to find out where is the problem in the code? Thanks for
any help.

What extension modules are you using?

I've never seen "stock" Python (stable release w/ only included modules)
segfault, but did see a segfault with an extension module I was using
the other week (lxml IIRC, but I'm not sure).

- Michael

[Maksim Kasimov]

there are a lot of packeges under suspicion:

cPickle, select, threading, logging, socket, struct
Michael Ekstrand wrote:

On Tuesday 04 October 2005 11:13, Maksim Kasimov wrote:

my programm sometime gives "Segmentation fault" message (no matter
how long the programm had run (1 day or 2 weeks). And there is
nothing in log-files that can points the problem. My question is how
it possible to find out where is the problem in the code? Thanks for
any help.

What extension modules are you using?

I've never seen "stock" Python (stable release w/ only included modules)
segfault, but did see a segfault with an extension module I was using
the other week (lxml IIRC, but I'm not sure).

- Michael

--
Best regards,
Maksim Kasimov
mailto: ma************@gmail.com

[Pierre Barbier de Reuille]

Maksim Kasimov a écrit :

Hello,

my programm sometime gives "Segmentation fault" message (no matter how
long the programm had run (1 day or 2 weeks). And there is nothing in
log-files that can points the problem.
My question is how it possible to find out where is the problem in the
code? Thanks for any help.

Python 2.2.3
FreeBSD

Well, your best bet is to generate a core file !
To do so, in the shell launching the program, you need to accept core
files. The command is :

$ ulimit -c <max size of core file accepted>

For example:
$ ulimit -c 500000

For a 500MB max file.

Then, if your program crash, you should see a file named "core.XXXX"
where XXXX is the PID of the process. You can know exactly where the
program crashed using gbd :

$ gdb --core=core.XXXX

Then, depending on the debug information you have in your executables
you may (or may not) be able to know what happened :)

Pierre

[Inyeol Lee]

On Tue, Oct 04, 2005 at 11:22:24AM -0500, Michael Ekstrand wrote:
[...]

I've never seen "stock" Python (stable release w/ only included modules)
segfault, but did see a segfault with an extension module I was using
the other week (lxml IIRC, but I'm not sure).

- Michael

So far, this is the simplest way to crash stock python, at least in
Unix/Linux;

$ python < /bin

If you redirect directory instead of file, python crashes. I think this bug was
introduced around 2.1 or 2.2, and not yet fixed as of 2.4.1.

- Inyeol

[Maksim Kasimov]

yes, to generete core dump is the best way,

but the command "$ ulimit -c 500000" don't make python to generete core dump in the time of crush.

I would like to know how to run python script so if it crushes than core dump will be genereted.

Thanks
Pierre Barbier de Reuille wrote:

Maksim Kasimov a écrit :

Hello,

my programm sometime gives "Segmentation fault" message (no matter how
long the programm had run (1 day or 2 weeks). And there is nothing in
log-files that can points the problem.
My question is how it possible to find out where is the problem in the
code? Thanks for any help.

Python 2.2.3
FreeBSD

Well, your best bet is to generate a core file !
To do so, in the shell launching the program, you need to accept core
files. The command is :

$ ulimit -c <max size of core file accepted>

For example:
$ ulimit -c 500000

For a 500MB max file.

Then, if your program crash, you should see a file named "core.XXXX"
where XXXX is the PID of the process. You can know exactly where the
program crashed using gbd :

$ gdb --core=core.XXXX

Then, depending on the debug information you have in your executables
you may (or may not) be able to know what happened :)

Pierre

--
Best regards,
Maksim Kasimov
mailto: ma************@gmail.com

[Pierre Barbier de Reuille]

Maksim Kasimov a écrit :

yes, to generete core dump is the best way,

but the command "$ ulimit -c 500000" don't make python to generete core
dump in the time of crush.

I would like to know how to run python script so if it crushes than core
dump will be genereted.

Thanks

If it does not, that probably means the core file is larger ... try a
larger value or even "unlimited" :

$ ulimit -c unlimited

Pierre

[Tamer Fahmy]

On Tue, 04 Oct 2005 19:13:10 +0300, Maksim Kasimov wrote:

my programm sometime gives "Segmentation fault" message (no matter how
long the programm had run (1 day or 2 weeks). And there is nothing in
log-files that can points the problem. My question is how it possible to
find out where is the problem in the code? Thanks for any help.

Python 2.2.3
FreeBSD

you could start your program within a gdb session like so:

$ gdb python
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc. GDB is free software,
covered by the GNU General Public License, and you are welcome to change
it and/or distribute copies of it under certain conditions. Type "show
copying" to see the conditions. There is absolutely no warranty for GDB.
Type "show warranty" for details. This GDB was configured as
"i386-marcel-freebsd"...(no debugging symbols found)...
(gdb) r foo.py
Starting program: /usr/bin/python foo.py ...

once your progmam segfaults you can then inspect the stack through:

(gdb) bt

cheers,
tamer.

--
hardware, n: The parts of a computer system that can be kicked.

[Franz Steinhaeusler]

On Wed, 05 Oct 2005 14:53:45 +0200, Tamer Fahmy
<ta***@remove-this-tammura.at-and.this> wrote:

On Tue, 04 Oct 2005 19:13:10 +0300, Maksim Kasimov wrote:

my programm sometime gives "Segmentation fault" message (no matter how
long the programm had run (1 day or 2 weeks). And there is nothing in
log-files that can points the problem. My question is how it possible to
find out where is the problem in the code? Thanks for any help.

Python 2.2.3
FreeBSD

you could start your program within a gdb session like so:

$ gdb python
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc. GDB is free software,
covered by the GNU General Public License, and you are welcome to change
it and/or distribute copies of it under certain conditions. Type "show
copying" to see the conditions. There is absolutely no warranty for GDB.
Type "show warranty" for details. This GDB was configured as
"i386-marcel-freebsd"...(no debugging symbols found)...
(gdb) r foo.py
Starting program: /usr/bin/python foo.py ...

once your progmam segfaults you can then inspect the stack through:

(gdb) bt

cheers,
tamer.

Tamer, thanks for your tip.

I tried Jeff's code:

import marshal
f = lambda: None
code =
marshal.loads(marshal.dumps(f.func_code).replace(' d\0\0S','d\xff\xffS'))
f.func_code = code
f()

saved it as foo.py, than

C:\Python24\Lib>gdb python
GNU gdb 5.1.1 (mingw experimental)
Copyright 2002 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you
are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for
details.
This GDB was configured as "mingw32"...(no debugging symbols found)...
(gdb) r foo.py
Starting program: C:\Python24/python.exe foo.py

Program received signal SIGSEGV, Segmentation fault.
0x1e027b23 in ?? ()
(gdb) bt
#0 0x1e027b23 in ?? ()
#1 0x00977240 in ?? ()
#2 0x1e1a82b8 in ?? ()
Cannot access memory at address 0x7
(gdb)

How can I interpret this results? ;)

Am I right, that I need a debug build of python built with mingw or
cygwin in Windows?
--
Franz Steinhaeusler

[Maksim Kasimov]

looks sad :(

ok, until the decision not found, i've run my script (if someone have the same problem) using sh-script like this:

#!/bin/sh
while true
do
myscript.py
echo "scrip was fault" | mail -s "run.sh" ma************@gmail.com
done

.... still works ;)
Franz Steinhaeusler wrote:

Tamer, thanks for your tip.

I tried Jeff's code:

import marshal
f = lambda: None
code =
marshal.loads(marshal.dumps(f.func_code).replace(' d\0\0S','d\xff\xffS'))
f.func_code = code
f()

saved it as foo.py, than

C:\Python24\Lib>gdb python
GNU gdb 5.1.1 (mingw experimental)
Copyright 2002 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you
are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for
details.
This GDB was configured as "mingw32"...(no debugging symbols found)...
(gdb) r foo.py
Starting program: C:\Python24/python.exe foo.py

Program received signal SIGSEGV, Segmentation fault.
0x1e027b23 in ?? ()
(gdb) bt
#0 0x1e027b23 in ?? ()
#1 0x00977240 in ?? ()
#2 0x1e1a82b8 in ?? ()
Cannot access memory at address 0x7
(gdb)

How can I interpret this results? ;)

Am I right, that I need a debug build of python built with mingw or
cygwin in Windows?

--
Best regards,
Maksim Kasimov
mailto: ma************@gmail.com