By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
459,717 Members | 1,233 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 459,717 IT Pros & Developers. It's quick & easy.

pyparsing and 'keywords'

P: n/a
Hello,

I'm having some problems with pyparsing, I could not find how to tell
it to view certain words as keywords, i.e. not as a possible variable
name (in an elegant way),
for example, I have this little grammar:

terminator = Literal(";")
expr = Word(alphas)
body = Forward();
ifstat = "if" + body + "fi"
stat = expr | ifstat
body << OneOrMore(stat + terminator)
program = body

I.e. some program which contains statements separated by semicolons. A
statement is either an if [....] fi statement or simply a word.

If I try however to parse the String "if test; testagain; fi;", it does
not work, because the fi is interpreted as an expr, not as the end of
the if statement, and of course, adding another fi doesn't solve this
either.

How to fix this?

Thank you,

Berteun

Jul 18 '05 #1
Share this Question
Share on Google+
3 Replies


P: n/a
"Berteun Damman" <be*****@gmail.com> wrote in message
news:11**********************@f14g2000cwb.googlegr oups.com...
Hello,

I'm having some problems with pyparsing, I could not find how to tell
it to view certain words as keywords, i.e. not as a possible variable
name (in an elegant way),
for example, I have this little grammar:

terminator = Literal(";")
expr = Word(alphas)
body = Forward();
ifstat = "if" + body + "fi"
stat = expr | ifstat
body << OneOrMore(stat + terminator)
program = body

I.e. some program which contains statements separated by semicolons. A
statement is either an if [....] fi statement or simply a word.

If I try however to parse the String "if test; testagain; fi;", it does
not work, because the fi is interpreted as an expr, not as the end of
the if statement, and of course, adding another fi doesn't solve this
either.

How to fix this?

Thank you,

Berteun

Berteun -

The simplest way I can think of for this grammar off the top of my head is
to use a parse action to reject keywords.

keywords = [ "if", "fi", "else", "return" ]
def rejectKeywords(string,loc,tokens):
if tokens[0] in keywords:
raise ParseException(string,loc,"found keyword %s" % tokens[0])
expr.setParseAction( rejectKeywords )

I took a different tack in the idl parser that is included in the pyparsing
examples directory, but it is more extensive.

-- Paul
Jul 18 '05 #2

P: n/a
On Tue, 14 Dec 2004 18:39:19 GMT, Paul McGuire
<pt***@austin.rr._bogus_.com> wrote:
If I try however to parse the String "if test; testagain; fi;", it does
not work, because the fi is interpreted as an expr, not as the end of
the if statement, and of course, adding another fi doesn't solve this
either.

The simplest way I can think of for this grammar off the top of my head is
to use a parse action to reject keywords.


Thank you for your quick and good solution, that did the trick indeed.

But I'm wondering, isn't it possible to have some sort of lexing phase
which already identifies keywords as such? Or to provide a table with
keywords, which pyparsing is able to automatically recognize?

So you would be able to do IF = Keyword("if"), just as a Literal now is
created, but now the parser knows this word shouldn't be interpreted any
other way than as a keyword. Or would that be a bad idea?

Berteun
Jul 18 '05 #3

P: n/a
"Berteun Damman" <berteun@NO_SPAMdds.nl> wrote in message
news:sl********************@cal005302.student.utwe nte.nl...
On Tue, 14 Dec 2004 18:39:19 GMT, Paul McGuire
<pt***@austin.rr._bogus_.com> wrote:
If I try however to parse the String "if test; testagain; fi;", it does
not work, because the fi is interpreted as an expr, not as the end of
the if statement, and of course, adding another fi doesn't solve this
either.

The simplest way I can think of for this grammar off the top of my head is to use a parse action to reject keywords.


Thank you for your quick and good solution, that did the trick indeed.

But I'm wondering, isn't it possible to have some sort of lexing phase
which already identifies keywords as such? Or to provide a table with
keywords, which pyparsing is able to automatically recognize?

So you would be able to do IF = Keyword("if"), just as a Literal now is
created, but now the parser knows this word shouldn't be interpreted any
other way than as a keyword. Or would that be a bad idea?

Berteun

Berteun -

This is not a bad idea in and of itself, as I mentioned earlier, I had
similar problems with the idl parser. For example, if in the body of your
if block you had a statement:

ifThisWorksItsAMiracle;

The leading "if" would match the Literal("if"), although the statement is
actually the complete 'ifThisWorksItsAMiracle' method call. That is,
Literals are not automatically assumed to be whitespace-terminated. This is
part of pyparsing's philosophy of being whitespace-blind.

It's been a while since v 1.2.2 was released, I think adding a Keyword class
much as you describe (on top of the other bug-fixes and additions since
then) would be sufficient material for coming up with a 1.3 release. I'll
work on it over the holiday "break," assuming I get one!

-- Paul
Jul 18 '05 #4

This discussion thread is closed

Replies have been disabled for this discussion.