473,883 Members | 1,626 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

embedding executable code in a regular expression in Python

Folks,

Does regular expression processing in Python allow for executable
code to be embedded inside a regular expression?

For example, in Perl the following two statements

$regex = qr/hello(?{print "saw hello\n"})mello (?{print "saw
mello\n"})/;
"jellohellomell o" =~ /$regex/;

will produce the output

saw hello
saw mello

Is it possible to do the same in Python with any modules that come
with the standard distribution, or with any other modules?

Thanks in advance for any help,

Avi Kak (ka*@purdue.edu )

Jul 16 '06 #1
5 2330

----- Original Message -----
From: "Avi Kak" <ka*@purdue.edu >
Newsgroups: comp.lang.pytho n
To: <py*********@py thon.org>
Sent: Sunday, July 16, 2006 11:05 PM
Subject: embedding executable code in a regular expression in Python

Folks,

Does regular expression processing in Python allow for executable
code to be embedded inside a regular expression?

For example, in Perl the following two statements

$regex = qr/hello(?{print "saw hello\n"})mello (?{print "saw
mello\n"})/;
"jellohellomell o" =~ /$regex/;

will produce the output

saw hello
saw mello

Is it possible to do the same in Python with any modules that come
with the standard distribution, or with any other modules?

Thanks in advance for any help,

Avi Kak (ka*@purdue.edu )

--
http://mail.python.org/mailman/listinfo/python-list
There's a new module out that can do it like this:

import SE
Stream_Editor = SE.SE ('<EAT"~[hm]ello~=print \'saw =\'\n" ')
for line in Stream_Editor ('yellomellojel lohello').split ('\n'):
exec ("%s" % line)

saw mello
saw hello

You'll find SE in the Cheese Shop. It has been commented favorably. I
wouldn't comment it, because I wrote it.

Regards

Frederic
Jul 16 '06 #2
On 2006-07-16, Avi Kak <ka*@purdue.edu wrote:
Folks,

Does regular expression processing in Python allow for executable
code to be embedded inside a regular expression?

For example, in Perl the following two statements

$regex = qr/hello(?{print "saw hello\n"})mello (?{print "saw
mello\n"})/;
"jellohellomell o" =~ /$regex/;

will produce the output

saw hello
saw mello

Is it possible to do the same in Python with any modules that come
with the standard distribution, or with any other modules?
You can use sub and make the replacement pattern a function (or any
"callable" thing) and it gets called back with the match object:

import re

def f(mo):
if "hello" in mo.groups():
print "saw hello"
if "mello" in mo.groups():
print "saw mello"

re.sub(r'(hello )(mello)', f, "jellohellomell o")

Actually I didn't know you could do that in Perl. The time I've found
this useful in Python is substitutions to convert e.g.
"background-color" into "backgroundColo r"; a function turns the c into
C. I always assumed in Perl you would need to use eval for this, but
perhaps there is another way.
Jul 16 '06 #3
Avi Kak wrote:
Does regular expression processing in Python allow for executable
code to be embedded inside a regular expression?

For example, in Perl the following two statements

$regex = qr/hello(?{print "saw hello\n"})mello (?{print "saw
mello\n"})/;
"jellohellomell o" =~ /$regex/;

will produce the output

saw hello
saw mello

Is it possible to do the same in Python with any modules that come
with the standard distribution, or with any other modules?
Just in case you were referring to security concerns: Sufficiently complex REs
can take ages to compile and run and eat tons of memory, so there are always
issues involved here.

Stefan
Jul 17 '06 #4
Avi Kak wrote:
Folks,

Does regular expression processing in Python allow for executable
code to be embedded inside a regular expression?

For example, in Perl the following two statements

$regex = qr/hello(?{print "saw hello\n"})mello (?{print "saw
mello\n"})/;
"jellohellomell o" =~ /$regex/;

will produce the output

saw hello
saw mello
Not nearly so terse, but perhaps easier to follow, here is a pyparsing
version. Pyparsing parse actions are intended to do just what you ask.
Parse actions may be defined to take no arguments, just one argument
(which will be passed the list of matching token strings), 2 arguments
(the match location and the matching tokens), or 3 arguments (the
original source string, the match location, and the tokens). Parse
actions are very good for transforming input text into modified output
form, such as the "background-color" to "backgroundColo r" transform -
the BoaConstructor team used pyparsing to implement a version upgrade
that transformed user source to a new version of wx (involving a
variety of suh changes).

Here is your jello/mello program, with two variations of parse actions.

-- Paul

from pyparsing import *

instr = "jellorelohello mellofellowbell owmello"
searchTerm = oneOf( ["jello","me llo"] )

# simple parse action, just echoes matched text
def echoMatchedText (tokens):
print "saw", tokens[0]

searchTerm.setP arseAction( echoMatchedText )
searchTerm.sear chString(instr)

# modified parse action, prints location too
def echoMatchedText (loc,tokens):
print "saw", tokens[0], "at locn", loc

searchTerm.setP arseAction( echoMatchedText )
searchTerm.sear chString(instr)

Prints out:
saw jello
saw mello
saw mello
saw jello at locn 0
saw mello at locn 14
saw mello at locn 31

Jul 17 '06 #5
Hi, see below ...

----- Original Message -----
From: "Paul McGuire" <pt***@austin.r r.com>
Newsgroups: comp.lang.pytho n
To: <py*********@py thon.org>
Sent: Monday, July 17, 2006 10:09 AM
Subject: Re: embedding executable code in a regular expression in Python

Avi Kak wrote:
Folks,

Does regular expression processing in Python allow for executable
code to be embedded inside a regular expression?

For example, in Perl the following two statements

$regex = qr/hello(?{print "saw hello\n"})mello (?{print "saw
mello\n"})/;
"jellohellomell o" =~ /$regex/;

will produce the output

saw hello
saw mello

Not nearly so terse, but perhaps easier to follow, here is a pyparsing
version. Pyparsing parse actions are intended to do just what you ask.
Parse actions may be defined to take no arguments, just one argument
(which will be passed the list of matching token strings), 2 arguments
(the match location and the matching tokens), or 3 arguments (the
original source string, the match location, and the tokens). Parse
actions are very good for transforming input text into modified output
form, such as the "background-color" to "backgroundColo r" transform -
the BoaConstructor team used pyparsing to implement a version upgrade
that transformed user source to a new version of wx (involving a
variety of suh changes).

Here is your jello/mello program, with two variations of parse actions.

-- Paul

from pyparsing import *

instr = "jellorelohello mellofellowbell owmello"
searchTerm = oneOf( ["jello","me llo"] )

# simple parse action, just echoes matched text
def echoMatchedText (tokens):
print "saw", tokens[0]

searchTerm.setP arseAction( echoMatchedText )
searchTerm.sear chString(instr)

# modified parse action, prints location too
def echoMatchedText (loc,tokens):
print "saw", tokens[0], "at locn", loc

searchTerm.setP arseAction( echoMatchedText )
searchTerm.sear chString(instr)

Prints out:
saw jello
saw mello
saw mello
saw jello at locn 0
saw mello at locn 14
saw mello at locn 31

--
http://mail.python.org/mailman/listinfo/python-list
One fine example of a pyparse solution!

On my part I had a second thought on the SE solution I proposed. It was
needlessly complicated. Let me try again:
>>sentence = 'On each occurrence of the word "square" in this text the
variable n will be squared. When it says "double" it will be doubled. At the
end print n % 11.'
>>def square_n (): global n; n *= n
def double_n (): global n; n += n
def n_mod_11 (): global n; return n % 11
se_definition s = """<EAT>
.... "square=pri nt 'Calling square_n () - ',; square_n (); print 'n is now
%d' % n \n" # A piece of code for 'square'
... "double=pri nt 'Calling double_n () - ',; double_n (); print 'n is now
%d' % n \n" # Another piece of code for 'double'
... ".=\nprint 'n %% 11 is: %d' % n_mod_11 ()\n" # Another piece of code
for the final dot
.... """
>>from SE import *
Stream_Edit or = SE (definitions)
n = 9; exec (Stream_Editor (sentence))
Calling square_n () - n is now 81
Calling square_n () - n is now 6561
n % 11 is: 5
Calling double_n () - n is now 13122
Calling double_n () - n is now 26244
n % 11 is: 9
n % 11 is: 9

Suppose we now realize that the quoted words "square" and "double" should
not trigger the respective function and neither should the ending dot of
each sentence, except the last one. We fix the defintions in seconds, adding
the exceptions as deletions. The order is immaterial. Targets may be regular
expressions. (The last one is.)

se_definitions = """<EAT # Deletes everything except the defined
substitutions
.... "square=pri nt 'Calling square_n () - ',; square_n (); print 'n is now
%d' % n \n" # A piece of code for 'square'
.... "double=pri nt 'Calling double_n () - ',; double_n (); print 'n is now
%d' % n \n" # Another piece of code for 'double'
.... ".=\nprint 'n %% 11 is: %d' % n_mod_11 ()\n" # Another piece of code
for the final dot
.... \""square"=" \""double"=" # Deletes the quoted words
.... "~\.\s~=" # Deletes dots followed by white space.
.... """
>>n = 9; exec (SE (se_definitions )(sentence))
Calling square_n () - n is now 81
Calling double_n () - n is now 162
n % 11 is: 8

Note 1: SE is no match for pyparse on many accounts. Conversely SE has its
own strengths in the realm of its specialty, in particular ease and speed of
use, versatility and modularity.
Most any problem can be solved in a variety of different ways. Finding
good approaches is one of the appeals of programming.

Note 2: The SE solution rests entirely on Python's 'exec' functionality and
so here we have another testimony to Python's excellence.

Note 3: I shall include the example in the manual. I hadn't thought of it.
My thanks to the OP for the inspiration. Perhaps he might want to explain
his purpose. I don't quite see the solution's problem.

Frederic
Jul 18 '06 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

18
2831
by: K_Lee | last post by:
I documented the regex internal implementation code for both Tcl and Python. As much as I like Tcl, I like Python's code much more. Tcl's Stub interface to the external commands is confusing to outsider. I still don't get why the stub interface is needed. One aspect I don't understanding about python is that the Python language itself is object oriented and all the internal is implement as C object. Why not C++ object?
4
2796
by: Alicia Haumann | last post by:
I accidentally sent this to webmaster@python.org, so this could be a duplicate if "webmaster" forwards it to this list. :{ Hi, there. Thanks for any help that can be offered. I've been working with Python for a year or more now, but only doing simple extending in C/C++. I'm now attempting some embedding and several questions have come to mind. BTW - I'm running Windows 2000 with Python23 and VisualC++ developers
1
1579
by: tom fogal | last post by:
Hi all, I can't seem to find out how to get a python script to run from a C (well, C++...) program. In particular, I'm confused about how the execution of the example code at http://docs.python.org/ext/pure-embedding.html works. If I call the example code 'multiply.py' and I generate an executable named 'call' (as the page suggests), I had expected the following to work: ../call multiply.py multiply 3 2
0
1116
by: Kenneth McDonald | last post by:
We're looking at embedding Python into our product to provide users with the ability to write scripts for the programming. My knowledge of Python is excellent, I'm familiar with the concepts of converting back and forth between C and Python datatypes, but my knowledge of compiling and linking is almost nonexistent. So if I could get advice on the following, it would be most appreciated. The program currently runs on Mac and Windows. 1)...
4
5704
by: David Abrahams | last post by:
I'm seeing highly surprising (and different!) behaviors of PyImport_ImportModule on Linux and Windows when used in a program with python embedding. On Linux, when attempting to import a module xxx that's in the current directory, I get ImportError: No module named xxx I can work around the problem by setting
4
2672
by: DavidM | last post by:
Hi all, I'm embedding python in a C prog which is built as a linux shared lib. The prog is linked against libpython, and on startup, it calls Py_Initialize(). The prog imports a pure-python script. The script starts up ok, but when it imports the 'math' module, the import fails with:
5
8298
by: Noah Hoffman | last post by:
I have been trying to write a regular expression that identifies a block of text enclosed by (potentially nested) parentheses. I've found solutions using other regular expression engines (for example, my text editor, BBEdit, which uses the PCRE library), but have not been able to replicate it using python's re module. Here's a version that works using the PCRE syntax, along with the python error message. I'm hoping for this to identify...
0
1563
by: gdetre | last post by:
Dear all, I'm trying to get a large, machine-generated regular expression (many thousands of characters) to work in Python on a Mac (running Leopard), and I keep banging my head against this brick wall: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Users/greg/elisp/freex/freex_sqlalchemy.py", line 715, in update_implicit_link_regexp_temp
8
3122
by: Uwe Schmitt | last post by:
Hi, Is anobody aware of this post: http://swtch.com/~rsc/regexp/regexp1.html ? Are there any plans to speed up Pythons regular expression module ? Or is the example in this artricle too far from reality ??? Greetings, Uwe
0
9943
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
11151
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
1
10858
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
9577
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
7974
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
7134
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5996
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
4619
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
4225
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.