473,545 Members | 1,829 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

overhead of starting threads

Hi,

What is the overhead of starting threads in Python? Currently my app
starts around 20 threads per second, but I'm considering creating a pool
of a fixed number of threads and keeping them fed with data. But only if
it would give me a noticeable performance boost.

Has anyone got any any experience in this area that they could share? -
before I start investigating myself..

I'm running on Windows, if thats relevant.
Regards,

Will McGugan

--
http://www.willmcgugan.com
"".join( [ {'*':'@','^':'. '}.get(c,None) or chr(97+(ord(c)-84)%26) for c
in "jvyy*jvyyzptht na^pbz" ] )
Jul 19 '05 #1
2 2105
How much you gain by starting threads is also determined by what you're
doing in those threads. Remember (or learn): In CPython only one thread
at a time can execute python code, so depending on your task threading
might gain you little. If you're doing I/O or calling functions written
in C (and if they release the Global Intepreter Lock [GIL]) you might
gain a lot by using threads.

As for overhead: profile, profile, profile. You'll have to do sample
runs and find your sweet-spot. It all depends on what you're doing in
the threads.

Jul 19 '05 #2
Simon Percivall wrote:
How much you gain by starting threads is also determined by what you're
doing in those threads. Remember (or learn): In CPython only one thread
at a time can execute python code, so depending on your task threading
might gain you little. If you're doing I/O or calling functions written
in C (and if they release the Global Intepreter Lock [GIL]) you might
gain a lot by using threads.
I wasn't asking about wether to use threads or not, I already am - for
several long running partialy I/O bound tasks.

As for overhead: profile, profile, profile. You'll have to do sample
runs and find your sweet-spot. It all depends on what you're doing in
the threads.


I haven't had much luck with the profile module in the standard library.
It only shows me data from the main thread. Is there a better way of
profile heavily threaded code?
Will McGugan

--
http://www.willmcgugan.com
"".join( [ {'*':'@','^':'. '}.get(c,None) or chr(97+(ord(c)-84)%26) for c
in "jvyy*jvyyzptht na^pbz" ] )
Jul 19 '05 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
1873
by: karl | last post by:
I have a windows service that kicks off a 'monitor' thread which in turn kicks off 4 additional threads. These 4 threads basically are listen on a designated socket and report back any errors (exceptions) to the monitor thread. Of course, all works well on my box. Yesterday I installed the .NET Framework 1.1 on a new server and then...
10
8754
by: Lenn | last post by:
Hello, I have always used a certain design pattern for multithreaded Windows app; Start new worker thread from UI thread, use events to notify UI threads when something happens, update UI controls using delegates through .BeginInvoke. I came across some code samples recently where ThreadPool class is used to start new threads. My...
1
1044
by: ErnieJ | last post by:
I a have a .net application which displays the results of a database query on a form in graphical format (i.e. charts and maps). This application is launched from another legacy Windows application with the query determined by a number of parameters. This calling application is not a .net app and does not support the type of display I am...
8
1980
by: raghu | last post by:
Hello Everyone I am working on C platform for a project in that project I need to maintain 'n' number of states where n is a dynamic value simultaneously. But I should not use any kind of threads since number of threads allowed, in the project, already reached and I can't predict how many threads to be maintained in this case. Can you...
0
7486
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main...
0
7416
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language...
0
7676
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. ...
1
7442
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For...
0
6001
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then...
1
5347
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes...
0
4965
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...
0
3456
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
0
729
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.