473,322 Members | 1,241 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,322 software developers and data experts.

Wierd problem with python sockets and threading

I have two python "applications" (more like scripts, they're only
about 80 lines each) that are dumbed down http-servers: They accept a
connection, reads everything in the socket untill "\r\n\r\n" and then
responds with "HTTP/1.1 200 OK\r\n\r\nHello World!" and then closes
the connection.

There is one multiplexing single-threaded one and one multithreaded
one, when I use apache bench with something like "ab -n 5000 -c 5
http://localhost/" everything works fine (about ~1500req/s each) - but
when I raise the concurrency (-c command in apache bench) of the bench
to something that is above the socket.listen(10) (ten in this case)
call in the server socket something strange happens: In the
multiplexing version everything still works fine, pulling ~1400req/s
or there about, but here is my problem:

In the multithreaded version of the code, when the concurrency level
in apache bench is raised above the socket.listen(10) call (say 11 or
100) performance dropps through the floor to something like ~10req/s.
But as long as the concurrency level stays less then or equal to the
socket.listen()-call everything works fine.

I've been staring at this problem for a day now, unable to figure it
out - I don't think I have any long locking times for the connection
queue used between the threads (and I can't see how they could occur
when the concurrency level is raised) and just can't see the reason.

Here's the code: http://paste2.org/p/89679 about 80 lines and it runs
as it is, no external deps. Any ideas/tips are greatly appriciated.

Regards, Fredrik.
Oct 20 '08 #1
0 698

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

6
by: Qun Cao | last post by:
Hello, I am just starting to play threading in python, here is a really interesting problem I am very curious about: " import thread def main(): thread.start_new(test.()) def test():
0
by: Samuel M. Smith | last post by:
I can build python 2.4.2 from source on the embedded linux box when I nfs mount and boot a full debian distribution. The embedded box also has stripped down linux distribution in onboard flash....
5
by: zxo102 | last post by:
Hi, I am doing a small project using socket server and thread in python. This is first time for me to use socket and thread things. Here is my case. I have 20 socket clients. Each client send a...
11
by: Krzysztof Retel | last post by:
Hi guys, I am struggling writing fast UDP server. It has to handle around 10000 UDP packets per second. I started building that with non blocking socket and threads. Unfortunately my approach...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.