Core dump with increased recursionlimit

[Nick Craig-Wood]

I've just discovered that my python (Python 2.3.4 from debian package
2.3.4-1 running on debian testing x86 + linux 2.4.26) core dumps when
I set recursionlimit very high and do lots of recursion.

Eg

$ python -c 'import sys; sys.setrecursionlimit(100000)
def f(): return f()
f()'
Segmentation fault (core dumped)

The recursion limit at which it does a core dump rather than a
"RuntimeError: maximum recursion depth exceeded" is about 7235 on my
system. (I binary searched it). However this number isn't constant!

Other interesting info

$ ulimit -a
core file size (blocks, -c) 1000000
data seg size (kbytes, -d) unlimited
file size (blocks, -f) unlimited
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
stack size (kbytes, -s) unlimited
cpu time (seconds, -t) unlimited
max user processes (-u) 7168
virtual memory (kbytes, -v) unlimited

So stack size should be unlimited.

The backtrace looks like this...

#0 0x400301ac in __pthread_alt_trylock () from /lib/libpthread.so.0
#1 0x40114003 in malloc () from /lib/libc.so.6
#2 0x080df577 in _PyObject_GC_Malloc ()
#3 0x080df67d in _PyObject_GC_NewVar ()
#4 0x080fb863 in PyFrame_New ()
#5 0x080ab8e1 in PyEval_CallObjectWithKeywords ()
#6 0x080ab6dc in PyEval_CallObjectWithKeywords ()
#7 0x080a9b9e in Py_MakePendingCalls ()
#8 0x080ab91d in PyEval_CallObjectWithKeywords ()
#9 0x080ab6dc in PyEval_CallObjectWithKeywords ()
#10 0x080a9b9e in Py_MakePendingCalls ()
#11 0x080ab91d in PyEval_CallObjectWithKeywords ()
#12 0x080ab6dc in PyEval_CallObjectWithKeywords ()
#13 0x080a9b9e in Py_MakePendingCalls ()

[snip many similar lines!]

#21713 0x080ab91d in PyEval_CallObjectWithKeywords ()
#21714 0x080ab6dc in PyEval_CallObjectWithKeywords ()
#21715 0x080a9b9e in Py_MakePendingCalls ()
#21716 0x080aa72c in PyEval_EvalCodeEx ()
#21717 0x080acf29 in PyEval_EvalCode ()
#21718 0x080d90ab in PyRun_FileExFlags ()
#21719 0x080d9c31 in PyRun_SimpleStringFlags ()
#21720 0x08054dc2 in Py_Main ()
#21721 0x080549eb in main ()

I don't have the debugging symbols for this version of python though
so can't provide more detail.

I'm pretty sure this isn't a limit of my linux setup. I compiled and
ran this C program

#include <stdio.h>
#include <string.h>

#define MB 256
#define SIZE (MB*1024*1024)

int main(void)
{
char blob[SIZE];
memset(blob, 0x55, SIZE);
printf("blob = %p, %d, %d\n", blob, blob[0], blob[SIZE-1]);
return 0;
}

Which runs fine and shows I can have 256 MB of stack. (Also the
python core dump is only 6 MB)

Any ideas? Problem with my setup? Linux bug? Python bug? Problem
between chair and keyboard?

--
Nick Craig-Wood <ni**@craig-wood.com> -- http://www.craig-wood.com/nick

[Michael Hoffman]

Nick Craig-Wood wrote:

I've just discovered that my python (Python 2.3.4 from debian package
2.3.4-1 running on debian testing x86 + linux 2.4.26) core dumps when
I set recursionlimit very high and do lots of recursion.
Man goes to a doctor and says "Doctor, it hurts when I do this!" The
doctor says, "Well, don't do that, then."

You might be interested in Stackless Python, which does not use the C
stack. I believe you could recurse until you run out of heap space.

http://www.stackless.com/
Which runs fine and shows I can have 256 MB of stack. (Also the
python core dump is only 6 MB)

Someone more knowledgable will have to explain this one for you <wink>.
--
Michael Hoffman

[Jeff Epler]

I thought I was getting similar behavior on
fedora-release-2-4
kernel-2.6.8-1.521
glibc-2.3.3-27
python-2.3.3-6
until I checked and saw that I had a 10240k soft limit for stack size.
When I change the limit to "unlimited" (ulimit -S -s unlimited) both the
Python program and C programs run to "completion" (a long traceback in
the Python program's case).

Jeff

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (GNU/Linux)

iD8DBQFBa9A9Jd01MZaTXX0RAqRlAKCPpDPtyemAh6TPXyZpA0 BJGlJ/7gCglC0I
FR39FgWa9kclyAiZUAzwYyw=
=VtGb
-----END PGP SIGNATURE-----

[Nick Craig-Wood]

<ex*****@divmod.com> wrote:

On 12 Oct 2004 09:29:58 GMT, Nick Craig-Wood <ni**@craig-wood.com> wrote:

I've just discovered that my python (Python 2.3.4 from debian package
2.3.4-1 running on debian testing x86 + linux 2.4.26) core dumps when
I set recursionlimit very high and do lots of recursion.

You probably have a buggy version of glibc. There is a version
which automatically sets the stack limit to a fairly small value (I
forget what the limit is exactly, 2 MB perhaps). If this is the
case and you strace your program, you will see a setrlimit() call
near the beginning of the run.

Spot on!

....
getrlimit(RLIMIT_STACK, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
setrlimit(RLIMIT_STACK, {rlim_cur=2044*1024, rlim_max=RLIM_INFINITY}) = 0
....

So its 1 page less than 2 MB.

Reading a bit more - this didn't affect my test C program because it
didn't use pthreads.

http://sources.redhat.com/ml/bug-gli.../msg00190.html

I'll look out for a glibc upgrade!

Thanks
--
Nick Craig-Wood <ni**@craig-wood.com> -- http://www.craig-wood.com/nick