By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
443,814 Members | 1,111 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 443,814 IT Pros & Developers. It's quick & easy.

Threading problem when many sockets open

P: n/a
Hi all,

I have written a socket based service in python and under fairly heavy
traffic it performs really well. But i have encountered the following
problem: when the system runs out of file descriptors, it seems to
stop switching control between threads.

Here is some more detail:

The system has n+2 threads, where n is usually around 10. This was
implemented using the 'threading' and 'socket' modules in the python
2.5 standard library.

-- The "master" thread accepts new socket connections and then
enqueues the connection on a Queue (from the standard library).

-- There are n "handler threads" that pop a connection off of the
queue, read some number of bytes (~10), do some processing and then
send ~100 bytes back over the connection and close it.

-- The last thread is just a logging thread that has a queue of
messages that it writes to either stdout or a file.

Under pretty substantial load, the processing is quick enough that the
connections do not pile up very quickly. But, in some cases they do.
And what I found was that as soon as the size of the Queue of
connections reached a high enough number (and it was always the same),
all of the processing seemed to stay in the "master" thread. I
created a siege client that opened up 1000+ connections and the server
would go into a state where the master thread repeatedly polled the
socket and printed an error. The Queue size stayed fixed at 997 even
if i shut down all of the connectionso n the client side. Under
normal operating conditions, those connections would be detected as
broken in the handler thread and a message would be logged. So it
seems that the "handler" threads aren't doing anything. (And they are
still alive, i check that in the master thread).

OK. I hope all of this is clear. Currently, I've solved the problem
by putting a cap on the queue size and i haven't seen the problem
reoccur. But it would be nice to understand exactly what went wrong.

Thanks in advance.

--
--
Philip Zigoris I SPOCK I 650.366.1165
Spock is Hiring!
www.spock.com/jobs
Aug 11 '07 #1
Share this Question
Share on Google+
1 Reply


P: n/a
In message <ma***************************************@python. org>, Philip
Zigoris wrote:
... and the server
would go into a state where the master thread repeatedly polled the
socket and printed an error.
Is that because it cannot create a new socket to accept the connection on?
Aug 25 '07 #2

This discussion thread is closed

Replies have been disabled for this discussion.