469,304 Members | 2,192 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,304 developers. It's quick & easy.

Custom behavior defined in the imported module

I have a new enhancement to pyparsing that doubles the parse speed (using a
technique called "packrat parsing"), but which is not suitable for all
parsers, specifically those that have complex parse actions. I don't want
to just enable this feature by default - I think there is too much risk of
it breaking existing code. Actually, I think the chance is <10%, but I
think I'm better off leaving the feature dormant unless explicitly enabled,
so that it doesn't take someone by surprise when they install the new
version.

The alternatives I've come up with for the user to enable this packrat parse
mode are:

1. Add a staticmethod enablePackrat() to the pyparsing ParserElement class,
to modify the ParserElement defintion of the internal (non-packrat) parse()
method. This method essentially runs code looking like:

ParserElement.parse,ParserElement.originalParse = \
ParserElement.packratParse,ParserElement.parse

where the packratParse method is a memoizing wrapper around calls to
originalParse. This technique requires the caller to invoke
ParserElement.enablePackrat() after having imported pyparsing.

This renaming trick actually breaks when using psyco, if psyco has compiled
the pyparsing methods before calling enablePackrat(). You have to import
pyparsing, call enablePackrat(), then call psyco.full(), or whatever psyco
commands to cache compiled versions of the pyparsing function code.

2. Implement some sort of "onImport" hook to allow the calling routine to
use a specialized import statement, something like "from pyparsing import
packratParsing" that would do the same as the above enablePackrat() call,
but perhaps in a more Pythonic-looking manner. (I was inspired along these
lines from Jim Hugunin's IronPython talk, in which he used import hooks to
modify the behavior of the base Python string class.) I've looked a bit at
imp, imputil, and ihooks, and these all seem to be modifiers to the import
mechanism as called from the importing module. What import hooks are
accessible from within the module being imported? Is there any way to see
which form of import was used, and to access the arguments of the import
command as invoked from the calling module?

I was hoping that by hooking into the import command, I could make this
mode-setting command more atomic with the actual import of pyparsing's class
and function definitions, to avoid problems like the psyco issue.
Plus I'm open to other suggestions as well.

Thanks,
-- Paul

Mar 25 '06 #1
4 1156
Paul McGuire wrote:
The alternatives I've come up with for the user to enable this packrat parse
mode are:

1. Add a staticmethod enablePackrat() to the pyparsing ParserElement class,
to modify the ParserElement defintion of the internal (non-packrat) parse()
method. This method essentially runs code looking like:

ParserElement.parse,ParserElement.originalParse = \
ParserElement.packratParse,ParserElement.parse


Could you just define a module pyparsingpackrat like this:
from pyparsing import *
ParserElement.parse,ParserElement.originalParse = \
ParserElement.packratParse,ParserElement.parse

Then users would just import pyparsingpackrat instead of pyparsing.

Kent
Mar 25 '06 #2
"Kent Johnson" <ke**@kentsjohnson.com> wrote in message
news:44**********@newspeer2.tds.net...
Paul McGuire wrote:
The alternatives I've come up with for the user to enable this packrat parse mode are:

1. Add a staticmethod enablePackrat() to the pyparsing ParserElement class, to modify the ParserElement defintion of the internal (non-packrat) parse() method. This method essentially runs code looking like:

ParserElement.parse,ParserElement.originalParse = \
ParserElement.packratParse,ParserElement.parse


Could you just define a module pyparsingpackrat like this:
from pyparsing import *
ParserElement.parse,ParserElement.originalParse = \
ParserElement.packratParse,ParserElement.parse

Then users would just import pyparsingpackrat instead of pyparsing.

Kent


I *really* like keeping pyparsing's footprint down
Mar 25 '06 #3
"Kent Johnson" <ke**@kentsjohnson.com> wrote in message
news:44**********@newspeer2.tds.net...
Paul McGuire wrote:
The alternatives I've come up with for the user to enable this packrat parse mode are:

1. Add a staticmethod enablePackrat() to the pyparsing ParserElement class, to modify the ParserElement defintion of the internal (non-packrat) parse() method. This method essentially runs code looking like:

ParserElement.parse,ParserElement.originalParse = \
ParserElement.packratParse,ParserElement.parse


Could you just define a module pyparsingpackrat like this:
from pyparsing import *
ParserElement.parse,ParserElement.originalParse = \
ParserElement.packratParse,ParserElement.parse

Then users would just import pyparsingpackrat instead of pyparsing.

Kent

(damned touchy touchpad!)

.... anyway...

I *really* like keeping pyparsing's footprint down to just one Python
module. It would be nice if I could pass an argument along with the import
statement:

import pyparsing(packrat=True)

or some such.

-- Paul
Mar 25 '06 #4
"Kent Johnson" <ke**@kentsjohnson.com> wrote in message
news:44**********@newspeer2.tds.net...
Paul McGuire wrote:
The alternatives I've come up with for the user to enable this packrat parse mode are:

1. Add a staticmethod enablePackrat() to the pyparsing ParserElement class, to modify the ParserElement defintion of the internal (non-packrat) parse() method. This method essentially runs code looking like:

ParserElement.parse,ParserElement.originalParse = \
ParserElement.packratParse,ParserElement.parse


Could you just define a module pyparsingpackrat like this:
from pyparsing import *
ParserElement.parse,ParserElement.originalParse = \
ParserElement.packratParse,ParserElement.parse

Then users would just import pyparsingpackrat instead of pyparsing.

Kent


I'm also not overkeen on elevating the word "packrat" into the module name
itself!

-- Paul
Mar 25 '06 #5

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

reply views Thread by paolo_veronelli | last post: by
5 posts views Thread by Tommytrojan | last post: by
19 posts views Thread by Jamey Shuemaker | last post: by
1 post views Thread by Kevin Walzer | last post: by
3 posts views Thread by Mohamed Yousef | last post: by
1 post views Thread by CARIGAR | last post: by
reply views Thread by suresh191 | last post: by
1 post views Thread by Geralt96 | last post: by
reply views Thread by harlem98 | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.