469,924 Members | 1,389 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,924 developers. It's quick & easy.

Regular expressions and the global interpreter lock


I have encountered a problem with the re module. I have a
multi-threaded program that does lots of regular expression searching,
with some relatively complex regular expressions. Occasionally, events
can conspire to mean that the re search takes minutes. That's bad
enough in and of itself, but the real problem is that the re engine
does not release the interpreter lock while it is running. All the
other threads are therefore blocked for the entire time it takes to do
the regular expression search.

Is there any fundamental reason why the re module cannot release the
interpreter lock, for at least some of the time it is running? The
ideal situation for me would be if it could do most of its work with
the lock released, since the software is running on a multi processor
machine that could productively do other work while the re is being
processed. Failing that, could it at least periodically release the
lock to give other threads a chance to run?

A quick look at the code in _sre.c suggests that for most of the time,
no Python objects are being manipulated, so the interpreter lock could
be released. Has anyone tried to do that?



-- Duncan Grisby --
-- du****@grisby.org --
-- http://www.grisby.org --
Nov 22 '05 #1
0 1191

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

3 posts views Thread by Tomas Christiansen | last post: by
7 posts views Thread by Patient Guy | last post: by
61 posts views Thread by km | last post: by
9 posts views Thread by Mike P | last post: by
7 posts views Thread by norton | last post: by
reply views Thread by Duncan Grisby | last post: by
4 posts views Thread by Egyd Csaba | last post: by
8 posts views Thread by Victor Polukcht | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.