473,597 Members | 2,092 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Threads, GIL and re.match() performance

Hi all

I understand that the C implementation of Python use a global interpreter
lock to avoid problems, so doing CPU bound tasks in multiple threads
will not result in better performance on multi-CPU systems.

However, I assumed that calls to (thread safe) C Library functions
release the global interpreter lock.

Today I checked the performance of some slow re.match() calls and found,
that the do not run in parallel on a multi-CPU system.

1) Is there a reason for this?
2) Is the regex library not thread-safe?
3) Is it possible, to release the GIL in re.match() to
get more performance?

I'm using Python 2.5

Thanks for your help

Mirko
--
"I've found that people who are great at something are not so much
convinced of their own greatness as mystified at why everyone else seems
so incompetent."
Paul Graham in "Great Hackers"
Jun 27 '08 #1
4 1346
In article <sl************ *************** *****@dziadzka. de>,
Mirko Dziadzka <mi************ @gmail.comwrote :
>
I understand that the C implementation of Python use a global interpreter
lock to avoid problems, so doing CPU bound tasks in multiple threads
will not result in better performance on multi-CPU systems.

However, I assumed that calls to (thread safe) C Library functions
release the global interpreter lock.
Generally speaking that only applies to I/O calls.
>Today I checked the performance of some slow re.match() calls and found,
that the do not run in parallel on a multi-CPU system.

1) Is there a reason for this?
2) Is the regex library not thread-safe?
3) Is it possible, to release the GIL in re.match() to
get more performance?
Theoretically possible, but the usual rule applies: patches welcome
--
Aahz (aa**@pythoncra ft.com) <* http://www.pythoncraft.com/

"as long as we like the same operating system, things are cool." --piranha
Jun 27 '08 #2
On Jun 25, 9:05*am, Mirko Dziadzka <mirko.dziad... @gmail.comwrote :
>
1) Is there a reason for this?
I think it is because the Python re library uses the Python C-API
which is not threadsafe.
2) Is the regex library not thread-safe?
3) Is it possible, to release the GIL in re.match() to
* *get more performance?
Jun 27 '08 #3
Hi,

The C-API uses references counts as well, so it is not threadsafe.

Matthieu

2008/6/26 Pau Freixes <pf******@gmail .com>:
But Python C-API[1] it's the main base for extent python with C/c++, and
this is not not threadsafe.? I dont understand

[1] http://docs.python.org/api/api.html

On Thu, Jun 26, 2008 at 4:49 AM, Benjamin <mu************ **@gmail.com>
wrote:
>>
On Jun 25, 9:05 am, Mirko Dziadzka <mirko.dziad... @gmail.comwrote :
>
1) Is there a reason for this?

I think it is because the Python re library uses the Python C-API
which is not threadsafe.
2) Is the regex library not thread-safe?
3) Is it possible, to release the GIL in re.match() to
get more performance?

--
http://mail.python.org/mailman/listinfo/python-list

--
Pau Freixes
Linux GNU/User
--
http://mail.python.org/mailman/listinfo/python-list


--
French PhD student
Website : http://matthieu-brucher.developpez.com/
Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92
LinkedIn : http://www.linkedin.com/in/matthieubrucher
Jun 27 '08 #4
However, I assumed that calls to (thread safe) C Library functions
release the global interpreter lock.
This is mainly applicable to external C libraries. The interface to
them may not be thread-safe; anything that uses the Python API to
create/manage Python objects will require use of the GIL. So the
actual regex search may release the GIL, but the storing of results
(and possibly intermediate results) would not.

Jun 27 '08 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

5
30662
by: Tzach | last post by:
I'm developing a simple Java client that runs over a CORBA server. The main client thread is waiting for notification from this server. On each notification, The client creates a new thread executing some logic with the server and close. Each session spend most of the time waiting for the server. At most there can be ~1000 open session. My questions are: 1. How many threads can be open at the same time (with response time of up to ~1...
5
537
by: Bart Nessux | last post by:
Could someone explain the concept of threads and how I might use them in Python? I was a math major, not a CS major (theory instead of practice). Most of my programming knowledge has grown out of system administration and shell scripting, not professional software development. How could I make this script threaded, and why should I, how would it benefit me? The script already takes up 100% CPU resources, what would using threads gain for...
1
4100
by: Dennis Gavrilov | last post by:
Hi, All! I have two questions: strategic and technical. Technical one first: I need to share an array of objects (implemented as hashes, having references to other objects and hashes, sharing done after blessing) between all of the mod_perl2 threads. The structure can grow quite big - tenths of thousands of array elements. It can grow as system operates (not possible to construct at apache startup). Sharing array is OK, but inserting...
6
2129
by: sathyashrayan | last post by:
Following are the selected thread from the date:30-jan-2005 to 31-jan-2005. I did not use any name because of the subject is important. You can get the original thread by typing the subject "string" in google comp.lang.c archives.Hope this helps.Hope I am not bothering any one. am I? =================================Start=========================== subject: Return to Start of Line? Question: I'd like printf, the next printf, to return...
34
10765
by: Kovan Akrei | last post by:
Hi, I would like to know how to reuse an object of a thread (if it is possible) in Csharp? I have the following program: using System; using System.Threading; using System.Collections; public class A {
7
3096
by: Michael | last post by:
I'm writing an application that decodes a file containing binary records. Each record is a particular event type. Each record is translated into ASCII and then written to a file. Each file contains the same events. At the moment each record is processed one after the other. It taks about 1m40s to process a large file containing 70,000 records. Would my application benifit from multiple threads and mmap? If so what is the best way to...
35
4006
by: Carl J. Van Arsdall | last post by:
Alright, based a on discussion on this mailing list, I've started to wonder, why use threads vs processes. So, If I have a system that has a large area of shared memory, which would be better? I've been leaning towards threads, I'm going to say why. Processes seem fairly expensive from my research so far. Each fork copies the entire contents of memory into the new process. There's also a more expensive context switch between...
18
2247
by: Jon Slaughter | last post by:
"Instead of just waiting for its time slice to expire, a thread can block each time it initiates a time-consuming activity in another thread until the activity finishes. This is better than spinning in a polling loop waiting for completion because it allows other threads to run sooner than they would if the system had to rely solely on expiration of a time slice to turn its attention to some other thread." I don't get the "a thread...
167
8239
by: darren | last post by:
Hi I have to write a multi-threaded program. I decided to take an OO approach to it. I had the idea to wrap up all of the thread functions in a mix-in class called Threadable. Then when an object should run in its own thread, it should implement this mix-in class. Does this sound like plausible design decision? I'm surprised that C++ doesn't have such functionality, say in its STL. This absence of a thread/object relationship in...
0
7965
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
7884
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
8380
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
8029
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
6683
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
3922
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
2394
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
1
1493
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
0
1230
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.