Threading server exits ERESTARTNOHAND

[Rune Hansen]

select(0, NULL, NULL, NULL, {0, 8000}) = 0 (Timeout)
futex(0x822f530, FUTEX_WAKE, 1) = 0
gettimeofday({1075823054, 603000}, NULL) = 0
futex(0x822f530, FUTEX_WAKE, 1) = 0
select(0, NULL, NULL, NULL, {0, 16000}) = 0 (Timeout)
futex(0x822f530, FUTEX_WAKE, 1) = 0
gettimeofday({1075823054, 623210}, NULL) = 0
futex(0x822f530, FUTEX_WAKE, 1) = 0
select(0, NULL, NULL, NULL, {0, 30370}) = ? ERESTARTNOHAND (To be restarted)
+++ killed by SIGSEGV +++

This is the output from strace -o xx python2.3 Spider.py, Spider.py
being a threaded web crawler. The server segfaults at no specific
interval, running for 30 secs and up to 10-15 minutes.

I can't figure out what's going on. In my years of threaded programming
this is the first time I've seen it (if it makes me a novice, please
don't rub it in :) I've found some information about "ERESTARTNOHAND"
(one hint was /etc/security/limits.conf) and gotten the feeling that it
has more to do with RedHat 9.0 than python2.3.3, but I'd really like to
hear from someone who perhaps has got some experience with this.

Server and setup:
Compiled Python2.3.3 on RedHat 9.0 with default Athlon kernel.

The server starts a XMLRPCServer and Controller. The Controller starts
15 threads doing urllib2.urlopen with data, using queue.get_nowait(),
received on the XMLRPCServer . Each thread has it's own MySQLdb
connection doing INSER DELAYED.
mxTidy is used before data is fed to an HTMLParser instance.

regards

/rune

[Rune Hansen]

Rune Hansen wrote:

select(0, NULL, NULL, NULL, {0, 8000}) = 0 (Timeout)
futex(0x822f530, FUTEX_WAKE, 1) = 0
gettimeofday({1075823054, 603000}, NULL) = 0
futex(0x822f530, FUTEX_WAKE, 1) = 0
select(0, NULL, NULL, NULL, {0, 16000}) = 0 (Timeout)
futex(0x822f530, FUTEX_WAKE, 1) = 0
gettimeofday({1075823054, 623210}, NULL) = 0
futex(0x822f530, FUTEX_WAKE, 1) = 0
select(0, NULL, NULL, NULL, {0, 30370}) = ? ERESTARTNOHAND (To be
restarted)
+++ killed by SIGSEGV +++

This is the output from strace -o xx python2.3 Spider.py, Spider.py
being a threaded web crawler. The server segfaults at no specific
interval, running for 30 secs and up to 10-15 minutes.

I can't figure out what's going on. In my years of threaded programming
this is the first time I've seen it (if it makes me a novice, please
don't rub it in :) I've found some information about "ERESTARTNOHAND"
(one hint was /etc/security/limits.conf) and gotten the feeling that it
has more to do with RedHat 9.0 than python2.3.3, but I'd really like to
hear from someone who perhaps has got some experience with this.

Server and setup:
Compiled Python2.3.3 on RedHat 9.0 with default Athlon kernel.

The server starts a XMLRPCServer and Controller. The Controller starts
15 threads doing urllib2.urlopen with data, using queue.get_nowait(),
received on the XMLRPCServer . Each thread has it's own MySQLdb
connection doing INSER DELAYED.
mxTidy is used before data is fed to an HTMLParser instance.

regards

/rune

In answer to my own question I'd like to quote our benevolent dictator:

"In any project that is multi-threaded, most bugs will come from
threading issues.
This is regardless of programming language -- it's a deep, as yet
ununderstood property of threads."
-- Guido van Rossum

So, there you are. Lesson learned.

regards

/rune