472,348 Members | 1,678 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 472,348 software developers and data experts.

Wierd problem with python sockets and threading

I have two python "applications" (more like scripts, they're only
about 80 lines each) that are dumbed down http-servers: They accept a
connection, reads everything in the socket untill "\r\n\r\n" and then
responds with "HTTP/1.1 200 OK\r\n\r\nHello World!" and then closes
the connection.

There is one multiplexing single-threaded one and one multithreaded
one, when I use apache bench with something like "ab -n 5000 -c 5
http://localhost/" everything works fine (about ~1500req/s each) - but
when I raise the concurrency (-c command in apache bench) of the bench
to something that is above the socket.listen(10) (ten in this case)
call in the server socket something strange happens: In the
multiplexing version everything still works fine, pulling ~1400req/s
or there about, but here is my problem:

In the multithreaded version of the code, when the concurrency level
in apache bench is raised above the socket.listen(10) call (say 11 or
100) performance dropps through the floor to something like ~10req/s.
But as long as the concurrency level stays less then or equal to the
socket.listen()-call everything works fine.

I've been staring at this problem for a day now, unable to figure it
out - I don't think I have any long locking times for the connection
queue used between the threads (and I can't see how they could occur
when the concurrency level is raised) and just can't see the reason.

Here's the code: http://paste2.org/p/89679 about 80 lines and it runs
as it is, no external deps. Any ideas/tips are greatly appriciated.

Regards, Fredrik.
Oct 20 '08 #1
0 677

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

by: Qun Cao | last post by:
Hello, I am just starting to play threading in python, here is a really interesting problem I am very curious about: " import thread def...
by: Samuel M. Smith | last post by:
I can build python 2.4.2 from source on the embedded linux box when I nfs mount and boot a full debian distribution. The embedded box also has...
by: zxo102 | last post by:
Hi, I am doing a small project using socket server and thread in python. This is first time for me to use socket and thread things. Here is my...
by: Krzysztof Retel | last post by:
Hi guys, I am struggling writing fast UDP server. It has to handle around 10000 UDP packets per second. I started building that with non blocking...
by: teenabhardwaj | last post by:
How would one discover a valid source for learning news, comfort, and help for engineering designs? Covering through piles of books takes a lot of...
by: Naresh1 | last post by:
What is WebLogic Admin Training? WebLogic Admin Training is a specialized program designed to equip individuals with the skills and knowledge...
by: jalbright99669 | last post by:
Am having a bit of a time with URL Rewrite. I need to incorporate http to https redirect with a reverse proxy. I have the URL Rewrite rules made...
by: Matthew3360 | last post by:
Hi there. I have been struggling to find out how to use a variable as my location in my header redirect function. Here is my code. ...
by: Matthew3360 | last post by:
Hi, I have a python app that i want to be able to get variables from a php page on my webserver. My python app is on my computer. How would I make it...
by: AndyPSV | last post by:
HOW CAN I CREATE AN AI with an .executable file that would suck all files in the folder and on my computerHOW CAN I CREATE AN AI with an .executable...
by: Arjunsri | last post by:
I have a Redshift database that I need to use as an import data source. I have configured the DSN connection using the server, port, database, and...
by: WisdomUfot | last post by:
It's an interesting question you've got about how Gmail hides the HTTP referrer when a link in an email is clicked. While I don't have the specific...
by: Oralloy | last post by:
Hello Folks, I am trying to hook up a CPU which I designed using SystemC to I/O pins on an FPGA. My problem (spelled failure) is with the...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.