What's the cost of using hundreds of threads?

Hello,

I have written some code, which creates many threads for each connection
('main connection'). The purpose of this code is to balance the load
between several connections ('pipes'). The number of spawned threads
depends on how many pipes I create (= 2*n+2, where n is the number of
pipes).

For good results I'll presumably share main connection's load between 10
pipes - therefore 22 threads will be spawned. Now if about 50
connections are forwarded the number of threads rises to thousand of
threads (or several thousands if even more connections are established).

My questions are:
- What is the cost (in memory / CPU usage) of creating such amounts of
threads?
- Is there any 'upper boundary' that limits the number of threads? (is
it python / OS related)
- Is that the sign of 'clumsy programming' - i.e. Is creating so many
threads a bad habit? (I must say that it simplified the solution of my
problem very much).

Limiting the number of threads is possible, but would affect the
independence of data flows. (ok I admit - creating tricky algorithm
could perhaps gurantee concurrency without spawning so many threads -
but it's the simplest solution to this problem :) ).

- Is there any 'upper boundary' that limits the number of threads? (is
it python / OS related)
- Is that the sign of 'clumsy programming' - i.e. Is creating so many
threads a bad habit? (I must say that it simplified the solution of my
problem very much).

Przemys³aw Ró¿ycki napisa³(a):

- Is there any 'upper boundary' that limits the number of threads? (is
it python / OS related)
- Is that the sign of 'clumsy programming' - i.e. Is creating so many
threads a bad habit? (I must say that it simplified the solution of my
problem very much).

I've read somewhere (I cann't recall where, though, was it MSDN?) that
Windows is not well suited to run more than 32 threads per process. Most
of the code I saw doesn't spawn more threads than a half of this.

I have written some code, which creates many threads for each connection
('main connection'). The purpose of this code is to balance the load
between several connections ('pipes'). The number of spawned threads
depends on how many pipes I create (= 2*n+2, where n is the number of
pipes).

For good results I'll presumably share main connection's load between 10
pipes - therefore 22 threads will be spawned. Now if about 50
connections are forwarded the number of threads rises to thousand of
threads (or several thousands if even more connections are established).
I'm a bit confused by your math. Fifty connections should be 102
threads, which is quite reasonable.
My questions are:
- What is the cost (in memory / CPU usage) of creating such amounts of
threads?
- Is there any 'upper boundary' that limits the number of threads? (is
it python / OS related)
- Is that the sign of 'clumsy programming' - i.e. Is creating so many
threads a bad habit? (I must say that it simplified the solution of my
problem very much).

Limiting the number of threads is possible, but would affect the
independence of data flows. (ok I admit - creating tricky algorithm
could perhaps gurantee concurrency without spawning so many threads -
but it's the simplest solution to this problem :) ).

threads, which is quite reasonable.
My formula applies to one forwarded ('loadbalanced' ) connection. Every
such connection creates further n connections (pipes) which share the
load. Every pipe requires two threads to be spawned. Every 'main
connection' spawns two other threads - so my formula: 2*pipes+2 gives
the number of threads spawned per 'main connection'.

Now if connections_cou nt connections are established the thread count
equals:
conn_count * threads_per_mai n_connection = conn_count * (2*pipes+2)

For 50 connections and about 10 pipes it will give 1100 threads.
My experience with lots of threads dates back to Python 1.5.2, but I
rarely saw much improvement with more than a hundred threads, even for
heavily I/O-bound applications on a multi-CPU system. However, if your
focus is algorithmic complexity, you should be able to handle a couple of
thousand threads easily enough.

Apache, for example, can easily spawn more threads under Windows, and
I've written code that uses 200 threads with excellent performance.
Things seem to slow down around the 2,000 mark for some reason I'm not
familiar with.

Thanks for your comments on winXP threads implementation. You confirmed
me in conviction that I shouldn't use windows.
Personally I use linux with 2.6.10 kernel, so hopefully I don't have to
share your grief. ;)