469,280 Members | 1,818 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,280 developers. It's quick & easy.

flex: plex?


Hi,

I am looking for a Python equivalent of Flex. After some
browsing, it seems that Plex is my best bet, but I would like to
hear suggestions before I dive in.

I have been using Flex a lot in combination with C programming.
I never use Yacc (or Bison), though Flex and Yacc are supposed
to be used together. But I found Yacc too cumbersome, and
unnecessary. Yacc is about context-free grammars, nice if you
like to describe an input language as a set of context-free
rewrite rules. Flex is about finite-state automata, but can be
extended to push-down automata (equivalent to context-free
grammars) and beyond, as long as it stays deterministic. I found
this to be very productive. I prefer automata over rewrite rules.

So, what I am looking for is a Python version of Flex that
supports the following capabilities, which are also supported by Plex:

- assigning any Python code to a matched input pattern

- input pattern recognition depending on 'states', which can be
changed through actions

Things that seems to be missing from Plex:

- patterns like:
A/B : match A only if followed by B, give A as result, keep B
in the input stream

- actions like:
REJECT: ask the flexer to come up with the next best match for the
same input
LESS(n): push n characters back into the input stream

All these are not really necessary, but they make some things
simpler.

So, are there any other packages that do these things better
than Plex?

--
Peter Kleiweg L:NL,af,da,de,en,ia,nds,no,sv,(fr,it) S:NL,de,en,(da,ia)
info: http://www.let.rug.nl/~kleiweg/ls.html

Jul 18 '05 #1
4 3380
If plex uses python regular expressions, you can use the zero-length
lookahead assertion instead of flex's A/B syntax: "A(?=B)"

Jeff

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFBInqKJd01MZaTXX0RAh5dAKCivbanEnAb1N7QShDi5U LE/+32DwCfTG34
cb0k64glgd54ryIoemvZBSo=
=l+Mj
-----END PGP SIGNATURE-----

Jul 18 '05 #2
Jeff Epler schreef:
If plex uses python regular expressions, you can use the zero-length
lookahead assertion instead of flex's A/B syntax: "A(?=B)"


Alas, it doesn't.
--
Peter Kleiweg L:NL,af,da,de,en,ia,nds,no,sv,(fr,it) S:NL,de,en,(da,ia)
info: http://www.let.rug.nl/~kleiweg/ls.html

Jul 18 '05 #3
At some point, Peter Kleiweg <in*************@nl.invalid> wrote:
Hi,

I am looking for a Python equivalent of Flex. After some
browsing, it seems that Plex is my best bet, but I would like to
hear suggestions before I dive in.

I have been using Flex a lot in combination with C programming.
I never use Yacc (or Bison), though Flex and Yacc are supposed
to be used together. But I found Yacc too cumbersome, and
unnecessary. Yacc is about context-free grammars, nice if you
like to describe an input language as a set of context-free
rewrite rules. Flex is about finite-state automata, but can be
extended to push-down automata (equivalent to context-free
grammars) and beyond, as long as it stays deterministic. I found
this to be very productive. I prefer automata over rewrite rules.


If you've been using Flex, you might want to check out FlexModule at
<http://www.cs.utexas.edu/users/mcguire/software/fbmodule/>
which allows you to use Flex to generate a lexer callable from python.

--
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke
|cookedm(at)physics(dot)mcmaster(dot)ca
Jul 18 '05 #4
David M. Cooke schreef:

If you've been using Flex, you might want to check out FlexModule at
<http://www.cs.utexas.edu/users/mcguire/software/fbmodule/>
which allows you to use Flex to generate a lexer callable from python.


I have tried it. (Had to edit the Makefile by hand, because
"make -f Makefile.pre.in boot" did not produce a valid Makefile,
but I did manage to build and run the example.) The problem is,
the Flex file (and Yacc file) is compiled into C, and then into
a shared lib. This means I must use C code in the Flex file.

What I want is something that lets me call arbitrary Python code
from the lexer. Ideal would be to have a version of Flex that
produces a Python file instead of a C file. I have kept looking,
and I think I found what I was looking for: PyLly, which is part
of PyGgy. http://www.lava.net/~newsham/pyggy/
--
Peter Kleiweg L:NL,af,da,de,en,ia,nds,no,sv,(fr,it) S:NL,de,en,(da,ia)
info: http://www.let.rug.nl/~kleiweg/ls.html

Jul 18 '05 #5

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

reply views Thread by srijit | last post: by
reply views Thread by Tom Heathcote | last post: by
2 posts views Thread by olig9 | last post: by
8 posts views Thread by pavel.orehov | last post: by
5 posts views Thread by ASP.NET explorer | last post: by
10 posts views Thread by Jules Winfield | last post: by
1 post views Thread by CARIGAR | last post: by
reply views Thread by zhoujie | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.