472,146 Members | 1,373 Online
Bytes | Software Development & Data Engineering Community
Post +

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 472,146 software developers and data experts.

Threading problem when many sockets open

Hi all,

I have written a socket based service in python and under fairly heavy
traffic it performs really well. But i have encountered the following
problem: when the system runs out of file descriptors, it seems to
stop switching control between threads.

Here is some more detail:

The system has n+2 threads, where n is usually around 10. This was
implemented using the 'threading' and 'socket' modules in the python
2.5 standard library.

-- The "master" thread accepts new socket connections and then
enqueues the connection on a Queue (from the standard library).

-- There are n "handler threads" that pop a connection off of the
queue, read some number of bytes (~10), do some processing and then
send ~100 bytes back over the connection and close it.

-- The last thread is just a logging thread that has a queue of
messages that it writes to either stdout or a file.

Under pretty substantial load, the processing is quick enough that the
connections do not pile up very quickly. But, in some cases they do.
And what I found was that as soon as the size of the Queue of
connections reached a high enough number (and it was always the same),
all of the processing seemed to stay in the "master" thread. I
created a siege client that opened up 1000+ connections and the server
would go into a state where the master thread repeatedly polled the
socket and printed an error. The Queue size stayed fixed at 997 even
if i shut down all of the connectionso n the client side. Under
normal operating conditions, those connections would be detected as
broken in the handler thread and a message would be logged. So it
seems that the "handler" threads aren't doing anything. (And they are
still alive, i check that in the master thread).

OK. I hope all of this is clear. Currently, I've solved the problem
by putting a cap on the queue size and i haven't seen the problem
reoccur. But it would be nice to understand exactly what went wrong.

Thanks in advance.

Philip Zigoris I SPOCK I 650.366.1165
Spock is Hiring!
Aug 11 '07 #1
1 1106
In message <ma***************************************@python. org>, Philip
Zigoris wrote:
... and the server
would go into a state where the master thread repeatedly polled the
socket and printed an error.
Is that because it cannot create a new socket to accept the connection on?
Aug 25 '07 #2

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

10 posts views Thread by Eric | last post: by
17 posts views Thread by Ryan Liu | last post: by
5 posts views Thread by Yehia A.Salam | last post: by
1 post views Thread by RFleming | last post: by
13 posts views Thread by Alexander Gnauck | last post: by
reply views Thread by Saiars | last post: by
reply views Thread by leo001 | last post: by

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.