473,395 Members | 1,456 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,395 software developers and data experts.

Core dump with increased recursionlimit

I've just discovered that my python (Python 2.3.4 from debian package
2.3.4-1 running on debian testing x86 + linux 2.4.26) core dumps when
I set recursionlimit very high and do lots of recursion.

Eg

$ python -c 'import sys; sys.setrecursionlimit(100000)
def f(): return f()
f()'
Segmentation fault (core dumped)

The recursion limit at which it does a core dump rather than a
"RuntimeError: maximum recursion depth exceeded" is about 7235 on my
system. (I binary searched it). However this number isn't constant!

Other interesting info

$ ulimit -a
core file size (blocks, -c) 1000000
data seg size (kbytes, -d) unlimited
file size (blocks, -f) unlimited
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
stack size (kbytes, -s) unlimited
cpu time (seconds, -t) unlimited
max user processes (-u) 7168
virtual memory (kbytes, -v) unlimited

So stack size should be unlimited.

The backtrace looks like this...

#0 0x400301ac in __pthread_alt_trylock () from /lib/libpthread.so.0
#1 0x40114003 in malloc () from /lib/libc.so.6
#2 0x080df577 in _PyObject_GC_Malloc ()
#3 0x080df67d in _PyObject_GC_NewVar ()
#4 0x080fb863 in PyFrame_New ()
#5 0x080ab8e1 in PyEval_CallObjectWithKeywords ()
#6 0x080ab6dc in PyEval_CallObjectWithKeywords ()
#7 0x080a9b9e in Py_MakePendingCalls ()
#8 0x080ab91d in PyEval_CallObjectWithKeywords ()
#9 0x080ab6dc in PyEval_CallObjectWithKeywords ()
#10 0x080a9b9e in Py_MakePendingCalls ()
#11 0x080ab91d in PyEval_CallObjectWithKeywords ()
#12 0x080ab6dc in PyEval_CallObjectWithKeywords ()
#13 0x080a9b9e in Py_MakePendingCalls ()

[snip many similar lines!]

#21713 0x080ab91d in PyEval_CallObjectWithKeywords ()
#21714 0x080ab6dc in PyEval_CallObjectWithKeywords ()
#21715 0x080a9b9e in Py_MakePendingCalls ()
#21716 0x080aa72c in PyEval_EvalCodeEx ()
#21717 0x080acf29 in PyEval_EvalCode ()
#21718 0x080d90ab in PyRun_FileExFlags ()
#21719 0x080d9c31 in PyRun_SimpleStringFlags ()
#21720 0x08054dc2 in Py_Main ()
#21721 0x080549eb in main ()

I don't have the debugging symbols for this version of python though
so can't provide more detail.

I'm pretty sure this isn't a limit of my linux setup. I compiled and
ran this C program

#include <stdio.h>
#include <string.h>

#define MB 256
#define SIZE (MB*1024*1024)

int main(void)
{
char blob[SIZE];
memset(blob, 0x55, SIZE);
printf("blob = %p, %d, %d\n", blob, blob[0], blob[SIZE-1]);
return 0;
}

Which runs fine and shows I can have 256 MB of stack. (Also the
python core dump is only 6 MB)

Any ideas? Problem with my setup? Linux bug? Python bug? Problem
between chair and keyboard?

--
Nick Craig-Wood <ni**@craig-wood.com> -- http://www.craig-wood.com/nick
Jul 18 '05 #1
3 6161
Nick Craig-Wood wrote:
I've just discovered that my python (Python 2.3.4 from debian package
2.3.4-1 running on debian testing x86 + linux 2.4.26) core dumps when
I set recursionlimit very high and do lots of recursion.
Man goes to a doctor and says "Doctor, it hurts when I do this!" The
doctor says, "Well, don't do that, then."

You might be interested in Stackless Python, which does not use the C
stack. I believe you could recurse until you run out of heap space.

http://www.stackless.com/
Which runs fine and shows I can have 256 MB of stack. (Also the
python core dump is only 6 MB)


Someone more knowledgable will have to explain this one for you <wink>.
--
Michael Hoffman
Jul 18 '05 #2
I thought I was getting similar behavior on
fedora-release-2-4
kernel-2.6.8-1.521
glibc-2.3.3-27
python-2.3.3-6
until I checked and saw that I had a 10240k soft limit for stack size.
When I change the limit to "unlimited" (ulimit -S -s unlimited) both the
Python program and C programs run to "completion" (a long traceback in
the Python program's case).

Jeff

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (GNU/Linux)

iD8DBQFBa9A9Jd01MZaTXX0RAqRlAKCPpDPtyemAh6TPXyZpA0 BJGlJ/7gCglC0I
FR39FgWa9kclyAiZUAzwYyw=
=VtGb
-----END PGP SIGNATURE-----

Jul 18 '05 #3
<ex*****@divmod.com> wrote:
On 12 Oct 2004 09:29:58 GMT, Nick Craig-Wood <ni**@craig-wood.com> wrote:
I've just discovered that my python (Python 2.3.4 from debian package
2.3.4-1 running on debian testing x86 + linux 2.4.26) core dumps when
I set recursionlimit very high and do lots of recursion.


You probably have a buggy version of glibc. There is a version
which automatically sets the stack limit to a fairly small value (I
forget what the limit is exactly, 2 MB perhaps). If this is the
case and you strace your program, you will see a setrlimit() call
near the beginning of the run.


Spot on!

....
getrlimit(RLIMIT_STACK, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
setrlimit(RLIMIT_STACK, {rlim_cur=2044*1024, rlim_max=RLIM_INFINITY}) = 0
....

So its 1 page less than 2 MB.

Reading a bit more - this didn't affect my test C program because it
didn't use pthreads.

http://sources.redhat.com/ml/bug-gli.../msg00190.html

I'll look out for a glibc upgrade!

Thanks
--
Nick Craig-Wood <ni**@craig-wood.com> -- http://www.craig-wood.com/nick
Jul 18 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: Martin | last post by:
I use dbx and i got the following error: Reading GL_CliConnMgr core file header read successfully Reading ld.so.1 dbx: core file read error: address 0xff3e6000 not available dbx: core file...
6
by: John Liu | last post by:
I've two questions, they may or may not be related - 1. I copied the entire data directory from postgreSQL 7.3.2 (AIX4.3) to the installation postgreSQL 7.3.4 (AIX5.1), the same filesystem setup....
10
by: ken | last post by:
hello, i'm writing a c program on a linux system. i'm debugging a segmentation fault but i don't want it to dump a core file because the memory footprint of the program is over 300Mb and i don't...
1
by: invincible | last post by:
hi, assume my program is running , is there a way I can dump its core in solaris, in linux i think u can use gcore Mohan
3
by: John Liu | last post by:
AIX pg version 7.4 Select * from document2 core dump. Did a few more experiments with select * from document2 limit... I limit to 500000 it works, 600000 it exits but says "calloc:...
4
by: madhusudan.hv | last post by:
hi, Can you tell me when an application might take long time to dump core? The linux o.s is indicating that a process is dumping core, and only after 40 mins i am seeing the core file. What might...
10
by: wong_powah | last post by:
I want to find out where (which line) my C program core dump. How to do that? Is there a web site describing the procedure? One approach is to use stack trace of the mdb debugger, but I does not...
14
by: Sheldon | last post by:
Hi, I have a python script that uses a C extention. I keep getting a recurring problem that causes a core dump a few lines after the C extention return data back tp python. I tried using pbd and...
5
by: johnericaturnbull | last post by:
Hi - I am very new to python. I get this random core dump and am looking for a good way to catch the error. I know the function my core dump occurs. Is there any error catching/handling that I...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.