By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
464,828 Members | 1,068 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 464,828 IT Pros & Developers. It's quick & easy.

Threads, GIL and re.match() performance

P: n/a
Hi all

I understand that the C implementation of Python use a global interpreter
lock to avoid problems, so doing CPU bound tasks in multiple threads
will not result in better performance on multi-CPU systems.

However, I assumed that calls to (thread safe) C Library functions
release the global interpreter lock.

Today I checked the performance of some slow re.match() calls and found,
that the do not run in parallel on a multi-CPU system.

1) Is there a reason for this?
2) Is the regex library not thread-safe?
3) Is it possible, to release the GIL in re.match() to
get more performance?

I'm using Python 2.5

Thanks for your help

Mirko
--
"I've found that people who are great at something are not so much
convinced of their own greatness as mystified at why everyone else seems
so incompetent."
Paul Graham in "Great Hackers"
Jun 27 '08 #1
Share this Question
Share on Google+
4 Replies

P: n/a
In article <sl********************************@dziadzka.de> ,
Mirko Dziadzka <mi************@gmail.comwrote:
>
I understand that the C implementation of Python use a global interpreter
lock to avoid problems, so doing CPU bound tasks in multiple threads
will not result in better performance on multi-CPU systems.

However, I assumed that calls to (thread safe) C Library functions
release the global interpreter lock.
Generally speaking that only applies to I/O calls.
>Today I checked the performance of some slow re.match() calls and found,
that the do not run in parallel on a multi-CPU system.

1) Is there a reason for this?
2) Is the regex library not thread-safe?
3) Is it possible, to release the GIL in re.match() to
get more performance?
Theoretically possible, but the usual rule applies: patches welcome
--
Aahz (aa**@pythoncraft.com) <* http://www.pythoncraft.com/

"as long as we like the same operating system, things are cool." --piranha
Jun 27 '08 #2

P: n/a
On Jun 25, 9:05*am, Mirko Dziadzka <mirko.dziad...@gmail.comwrote:
>
1) Is there a reason for this?
I think it is because the Python re library uses the Python C-API
which is not threadsafe.
2) Is the regex library not thread-safe?
3) Is it possible, to release the GIL in re.match() to
* *get more performance?
Jun 27 '08 #3

P: n/a
Hi,

The C-API uses references counts as well, so it is not threadsafe.

Matthieu

2008/6/26 Pau Freixes <pf******@gmail.com>:
But Python C-API[1] it's the main base for extent python with C/c++, and
this is not not threadsafe.? I dont understand

[1] http://docs.python.org/api/api.html

On Thu, Jun 26, 2008 at 4:49 AM, Benjamin <mu**************@gmail.com>
wrote:
>>
On Jun 25, 9:05 am, Mirko Dziadzka <mirko.dziad...@gmail.comwrote:
>
1) Is there a reason for this?

I think it is because the Python re library uses the Python C-API
which is not threadsafe.
2) Is the regex library not thread-safe?
3) Is it possible, to release the GIL in re.match() to
get more performance?

--
http://mail.python.org/mailman/listinfo/python-list

--
Pau Freixes
Linux GNU/User
--
http://mail.python.org/mailman/listinfo/python-list


--
French PhD student
Website : http://matthieu-brucher.developpez.com/
Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92
LinkedIn : http://www.linkedin.com/in/matthieubrucher
Jun 27 '08 #4

P: n/a
However, I assumed that calls to (thread safe) C Library functions
release the global interpreter lock.
This is mainly applicable to external C libraries. The interface to
them may not be thread-safe; anything that uses the Python API to
create/manage Python objects will require use of the GIL. So the
actual regex search may release the GIL, but the storing of results
(and possibly intermediate results) would not.

Jun 27 '08 #5

This discussion thread is closed

Replies have been disabled for this discussion.