Best lex/yacc for C++?

[Moonlit]

Hi,

I am searching for the best lex and yacc combination (or something similar)
that can be used in combination with C++ and that can contain C++ code. I
have the regular flex/bison port working but then I, of course, can't use
C++ constructs in the actions.

I have tried spirit although I could make a simple command line parser
(after too many hours), creating a parser tree with it was not possible in
an easy way (the compiler choked while compiling the include files), so back
to a lex/yacc clone I guess.

Does anyone have any opinion (especially if you have or are using some
implemtation yourself).

Any help appreciated.

Regards, Ron AF Greve.

[Kris Wempa]

"Moonlit" <al******@jupiter.universe> wrote in message
news:3f***********************@news.xs4all.nl...

Hi,

I am searching for the best lex and yacc combination (or something similar) that can be used in combination with C++ and that can contain C++ code. I
have the regular flex/bison port working but then I, of course, can't use
C++ constructs in the actions.

You can write "wrapper" functions that link to and call the actual C++ code.
What C++ code are you looking to use in the actions ?

[tom_usenet]

On Wed, 8 Oct 2003 17:30:56 +0200, "Moonlit"
<al******@jupiter.universe> wrote:

Hi,

I am searching for the best lex and yacc combination (or something similar)
that can be used in combination with C++ and that can contain C++ code. I
have the regular flex/bison port working but then I, of course, can't use
C++ constructs in the actions.

I have tried spirit although I could make a simple command line parser
(after too many hours), creating a parser tree with it was not possible in
an easy way (the compiler choked while compiling the include files), so back
to a lex/yacc clone I guess.

There is a spirit mailing list I believe:

https://lists.sourceforge.net/lists/...spirit-general

Compatibility information is here:

http://www.boost.org/libs/spirit/doc/portability.html

Tom

[Moonlit]

Hi,

Thanks for your reply.

Ah, they should have linked that on their front page I think. Now I see my
vc6 is apparently
not supported (well not the parse trees, but without them ....).

I tried actually their news server but for some reason my news post won't
arrive there though I can read the other posts.

Has someone got experience with some kind of language parser (doesn't have
to be big but at least with more than say 20 rules) with vc7.1 and spirit.

I must admit the way you can write your parser just in C++ code appeals to
me a lot, that's why I spend already so much time on it and I would be
willing to buy the 7.1 compiler just for that, if I would be at least
recently confident that I could use it then (with parse trees of reasonable
size).

Thanks for your reply again.
Regards, Ron AF Greve.



"tom_usenet" <to********@hotmail.com> wrote in message
news:44********************************@4ax.com...

On Wed, 8 Oct 2003 17:30:56 +0200, "Moonlit"
<al******@jupiter.universe> wrote:

Hi,

I am searching for the best lex and yacc combination (or something similar)that can be used in combination with C++ and that can contain C++ code. I
have the regular flex/bison port working but then I, of course, can't use
C++ constructs in the actions.

I have tried spirit although I could make a simple command line parser
(after too many hours), creating a parser tree with it was not possible inan easy way (the compiler choked while compiling the include files), so backto a lex/yacc clone I guess.

There is a spirit mailing list I believe:

https://lists.sourceforge.net/lists/...spirit-general

Compatibility information is here:

http://www.boost.org/libs/spirit/doc/portability.html

Tom

[Moonlit]

Hi,

Well amongst other things I would like to fill STL map templates with thing
like var[ "name" ] = etc, also I would call some classes to log info like
"Output << Debuginfo << End;" (the "End" is not a typo). But of course other
things too. In the past I just used a flex/bison scanner and then afterwards
I parsed the tree and created a tree of classes of it. Yes, that works and
it is easy to write but it feals like doing everything twice.

Actually I must admit I have never tried to call C++ from C I know of course
your can wrap stuff like in extern "C" { FILE* yyin; }. Would extern "C++"
{ CWhatEver *ClassFactory(); } let you do that the other way around? It
would be a solution although of course you get a lot of type casting and
loose the type checking.

Well its worth trying if there isn't a better solution.

Thanks for your reply.

Regards, Ron AF Greve.

"Kris Wempa" <calmincents(NO_SPAM)@yahoo.com> wrote in message
news:bm**********@kcweb01.netnews.att.com...

"Moonlit" <al******@jupiter.universe> wrote in message
news:3f***********************@news.xs4all.nl...

Hi,

I am searching for the best lex and yacc combination (or something similar)

that can be used in combination with C++ and that can contain C++ code. I have the regular flex/bison port working but then I, of course, can't use C++ constructs in the actions.

You can write "wrapper" functions that link to and call the actual C++

code. What C++ code are you looking to use in the actions ?

[Rob Williscroft]

Moonlit wrote in news:3f***********************@news.xs4all.nl:

Ah, they should have linked that on their front page I think. Now I
see my vc6 is apparently
not supported (well not the parse trees, but without them ....).

I tried actually their news server but for some reason my news post
won't arrive there though I can read the other posts.
I belive you have too register for there mailing list's before you
can post.

http://www.boost.org/more/mailing_lists.htm

Has someone got experience with some kind of language parser (doesn't
have to be big but at least with more than say 20 rules) with vc7.1
and spirit.

I must admit the way you can write your parser just in C++ code
appeals to me a lot, that's why I spend already so much time on it and
I would be willing to buy the 7.1 compiler just for that, if I would
be at least recently confident that I could use it then (with parse
trees of reasonable size).

Last time I checked the .NET SDK it came with a version (comandline only)
of the 7.1 compiler.

Also see/ask in: news://comp.compilers from what I see the topic is
mostly parsing.
Rob.
--
http://www.victim-prime.dsl.pipex.com/

[Kris Wempa]

"Moonlit" <al******@jupiter.universe> wrote in message
news:3f***********************@news.xs4all.nl...

Hi,

Well amongst other things I would like to fill STL map templates with thing like var[ "name" ] = etc, also I would call some classes to log info like
"Output << Debuginfo << End;" (the "End" is not a typo). But of course other things too. In the past I just used a flex/bison scanner and then afterwards I parsed the tree and created a tree of classes of it. Yes, that works and
it is easy to write but it feals like doing everything twice.

The only way you can use maps this way is to either:

1) create a map instance in a C++ compiler, write C functions to call the
map methods, give these C functions C linkage with: extern "C" {} and call
them from the flex/bison generated C program.

2) write the C only flex/bison code and generate the .c files, manually
edit the .c files to insert your C++ code, compile the resulting code with a
C++ compiler

Step 2 can be messy if your grammar is very large. I haven't looked at
flex/bison output in a long time, so I don't remember how ugly it can get.
Step 1 is easier, but you will also lose the ease of using the maps
directly. You'd essentially be calling a wrapper insert() function instead
of simply assigning the field/value like you have shown above. In other
words:

var[(string) "field"] = ((string) "value");

would become something like

mapinsert("field","value");

I hope these give you some ideas.

[Kris Wempa]

"Kris Wempa" <calmincents(NO_SPAM)@yahoo.com> wrote in message
news:bm**********@kcweb01.netnews.att.com...

1) create a map instance in a C++ compiler, write C functions to call the

Sorry, that should be "in a C++ file" .... not compiler

[Xenos]

"Moonlit" <al******@jupiter.universe> wrote in message
news:3f***********************@news.xs4all.nl...

Hi,

I am searching for the best lex and yacc combination (or something similar) that can be used in combination with C++ and that can contain C++ code. I
have the regular flex/bison port working but then I, of course, can't use
C++ constructs in the actions.

I absolutely love ANLTr. It will generate Java, C++, and C#. I use it for
C++. It has a steep learning curve, but is very powerful. It will generate
LL(k) grammars, and has a lot of neat features like semantic and syntactic
predicates. It generates surprising tight code--looks like a hand-generated
parser. The parse and lexer creation process is well integrated.
http://www.antlr.org
DrX.

[Moonlit]

Hi,

I didn't actually know you could call c from c++ that way, so thanks for the
ideas. However I usually build a tree from the input. In C I creates structs
(with a number assigned so I know what it represents) in most actions an
return pointers to it, what I end up then is a tree that represents the
input. Though it is not impossible to do the same using your first method, I
think it would be not easy to maintain but indeed you would only have to
build a tree once unlike twice as I do it now (first C then convert it to
C++).

Editing the generated code I would rather not, at some point you always
change the grammar and have to redo the changes.

Thanks for the reply.

Regards, Ron AF Greve.
"Kris Wempa" <calmincents(NO_SPAM)@yahoo.com> wrote in message
news:bm**********@kcweb01.netnews.att.com...

"Moonlit" <al******@jupiter.universe> wrote in message
news:3f***********************@news.xs4all.nl...

Hi,

Well amongst other things I would like to fill STL map templates with thing

like var[ "name" ] = etc, also I would call some classes to log info like "Output << Debuginfo << End;" (the "End" is not a typo). But of course

other

things too. In the past I just used a flex/bison scanner and then

afterwards

I parsed the tree and created a tree of classes of it. Yes, that works and it is easy to write but it feals like doing everything twice.

The only way you can use maps this way is to either:

1) create a map instance in a C++ compiler, write C functions to call the
map methods, give these C functions C linkage with: extern "C" {} and call
them from the flex/bison generated C program.

2) write the C only flex/bison code and generate the .c files, manually
edit the .c files to insert your C++ code, compile the resulting code with

a C++ compiler

Step 2 can be messy if your grammar is very large. I haven't looked at
flex/bison output in a long time, so I don't remember how ugly it can get.
Step 1 is easier, but you will also lose the ease of using the maps
directly. You'd essentially be calling a wrapper insert() function instead of simply assigning the field/value like you have shown above. In other
words:

var[(string) "field"] = ((string) "value");

would become something like

mapinsert("field","value");

I hope these give you some ideas.

[Moonlit]

Hi,

"Xenos" <do**********@spamhate.com> wrote in message
news:bm*********@cui1.lmms.lmco.com...

"Moonlit" <al******@jupiter.universe> wrote in message
news:3f***********************@news.xs4all.nl...

Hi,

I am searching for the best lex and yacc combination (or something similar)

that can be used in combination with C++ and that can contain C++ code. I have the regular flex/bison port working but then I, of course, can't use C++ constructs in the actions.

I absolutely love ANLTr. It will generate Java, C++, and C#. I use it

for C++. It has a steep learning curve, but is very powerful. It will generate LL(k) grammars, and has a lot of neat features like semantic and syntactic
predicates. It generates surprising tight code--looks like a hand-generated parser. The parse and lexer creation process is well integrated.
Ok, I think I have some reading to then. I already went over all the spirit
documentation. And despite that my VC6 compiler, generated invalid object
files, stack overlows and internal compiler errors.. well apart from that I
liked it.

But I think I just give antlr a try as well.

Thanks for the link and the reply.

Regards, Ron AF Greve.

http://www.antlr.org
DrX.

[David B. Held]

"Moonlit" <al******@jupiter.universe> wrote in message
news:3f***********************@news.xs4all.nl...

[...]
I must admit the way you can write your parser just in C++
code appeals to me a lot, that's why I spend already so
much time on it and I would be willing to buy the 7.1
compiler just for that, if I would be at least recently
confident that I could use it then (with parse trees of
reasonable size).

If you're willing to switch compilers, try gcc/cygwin.
Unless your grammar is very large. Then compile times
or instantiation depth might slay you. You can always
break your grammar up into parts, which helps, but Spirit
can't really compete with traditional compiler compilers
for very large projects. For small and medium size stuff,
though, it's very very sweet. I'm using the ASTs with gcc,
and while they aren't as well-developed as they could be,
it's amazing how much power you get from just a few
lines of code. It's almost like coding in VB without all
the nasty stuff you get with VB.

Dave

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.521 / Virus Database: 319 - Release Date: 9/23/2003

[Moonlit]

Hi,

"David B. Held" <dh***@codelogicconsulting.com> wrote in message
news:bm**********@news.astound.net...

"Moonlit" <al******@jupiter.universe> wrote in message
news:3f***********************@news.xs4all.nl...

[...]
I must admit the way you can write your parser just in C++
code appeals to me a lot, that's why I spend already so
much time on it and I would be willing to buy the 7.1
compiler just for that, if I would be at least recently
confident that I could use it then (with parse trees of
reasonable size).
If you're willing to switch compilers, try gcc/cygwin.
Unless your grammar is very large. Then compile times
or instantiation depth might slay you. You can always
break your grammar up into parts, which helps, but Spirit
can't really compete with traditional compiler compilers
for very large projects. For small and medium size stuff,
though, it's very very sweet. I'm using the ASTs with gcc,
and while they aren't as well-developed as they could be,
it's amazing how much power you get from just a few
lines of code. It's almost like coding in VB without all
the nasty stuff you get with VB.

Yes, it appeared to me that with spirit you could do things with just a few
lines of code and without the hassle of first compiling with bison and flex.
Unfortunately the ast stuff doesn't work for vc6.

I tried the gcc/cygwin solution one day (I do use g++ for unix platforms),
many years ago and things might have improved. But at that time, though I
could create a window, it really was lacking the IDE that goes with gcc. I
know there is ddd and such but VC6's IDE is real easy to use. It really
reduces the development time a lot (not to mention all the documentation
that you get with it)

Thanks for your reply,

Ron AF Greve.

Dave

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.521 / Virus Database: 319 - Release Date: 9/23/2003

[Jerry Coffin]

In article <3f***********************@news.xs4all.nl>,
al******@jupiter.universe says...

Hi,

I am searching for the best lex and yacc combination (or something similar)
that can be used in combination with C++ and that can contain C++ code. I
have the regular flex/bison port working but then I, of course, can't use
C++ constructs in the actions.

Flex (at least in recent versions) produces output that can be compiled
as C++. In fact, with the "-+" flag, it'll produce real C++ output
(i.e. the parser as a class instead of just C that happens to be
compilable as C++).

BYACC produces output that can be compiled as C++, and I've used C++
specific "stuff" in actions, including using a map for a symbol table,
much as you've mentioned.

One thing I should point out is that when I'm using C++, I generally do
not use %union and the built-in type system -- instead, I normally use
the C++ type system, typically with a polymorphic type.

--
Later,
Jerry.

The universe is a figment of its own imagination.

[Moonlit]

Hi,

I knew some flex implementations could do C++ (and some bison). However I
hoped there was some strong opionion on what to use on what not.

Apparently, there isn't one very good solution. I haven't made up my mind
yet but willl add byacc to it.

For mysef I made the following list:

spririt very tight code, is C++ code, no extra dependecies, know how it
works, unfortunately only got very small, simple command line parsers
working (migh be due to my VC6 version, certainly have to buy vc7.1 then)

Antlr Don't know anything about this, seems widely supported for different
languages, user java to generate code (I think).

Flex++/Bison++/byacc different versions on the net, lot of experience with
flex/bison so probably the most easy to use, more code than with spirit but
very likely more compatible with any compiler arround..

Hmmm, still difficult to choose, ok at some point I just have to pick one I
guess.

I just decided to order my copy of vc++ 2003, hope it works with spirit and
well, I could always backtrack to flex++/byacc/bison++ if not.
Thanks for your help.

Regards, Ron AF Greve.
"Jerry Coffin" <jc*****@taeus.com> wrote in message
news:MP************************@news.clspco.adelph ia.net...

In article <3f***********************@news.xs4all.nl>,
al******@jupiter.universe says...

Hi,

I am searching for the best lex and yacc combination (or something similar) that can be used in combination with C++ and that can contain C++ code. I have the regular flex/bison port working but then I, of course, can't use C++ constructs in the actions.

Flex (at least in recent versions) produces output that can be compiled
as C++. In fact, with the "-+" flag, it'll produce real C++ output
(i.e. the parser as a class instead of just C that happens to be
compilable as C++).

BYACC produces output that can be compiled as C++, and I've used C++
specific "stuff" in actions, including using a map for a symbol table,
much as you've mentioned.

One thing I should point out is that when I'm using C++, I generally do
not use %union and the built-in type system -- instead, I normally use
the C++ type system, typically with a polymorphic type.

--
Later,
Jerry.

The universe is a figment of its own imagination.

[Xenos]

"Moonlit" <al******@jupiter.universe> wrote in message
news:3f***********************@news.xs4all.nl...

Hi,

Antlr Don't know anything about this, seems widely supported for different languages, user java to generate code (I think).
yes it is written in Java.
Hmmm, still difficult to choose, ok at some point I just have to pick one I guess.
You might also check out :
PCCTS -- This the predecessor to ANLTr, but it is written in C++.

There is a nice list of compiler compilers here:
http://catalog.compilertools.net

I just decided to order my copy of vc++ 2003, hope it works with spirit and well, I could always backtrack to flex++/byacc/bison++ if not.

The only problem I see with this is the latest version of VC++ still has
weak template support, and as you already know, Spirit is very template
intensive. Have you checked out the BOOST library, to which Spirit was
resently added too? It has some nice workarounds for various compilers.
DrX

[Moonlit]

Hi,

"Xenos" <do**********@spamhate.com> wrote in message
news:bm*********@cui1.lmms.lmco.com...

"Moonlit" <al******@jupiter.universe> wrote in message
news:3f***********************@news.xs4all.nl...

Hi,

Antlr Don't know anything about this, seems widely supported for different

languages, user java to generate code (I think).

yes it is written in Java.

Hmmm, still difficult to choose, ok at some point I just have to pick one I

guess.
You might also check out :
PCCTS -- This the predecessor to ANLTr, but it is written in C++.

There is a nice list of compiler compilers here:
http://catalog.compilertools.net

Thanks, I will check them out.

I just decided to order my copy of vc++ 2003, hope it works with spirit

and

well, I could always backtrack to flex++/byacc/bison++ if not.

The only problem I see with this is the latest version of VC++ still has
weak template support, and as you already know, Spirit is very template
intensive. Have you checked out the BOOST library, to which Spirit was
resently added too? It has some nice workarounds for various compilers.

Well they say over in the spirit newsgroup that vc7.1 compiles very well,
but of course, the proof of the...

I tried different versions of spirit 1.6.1 (the latest stable release).
Could create a simple parser but when I wanted to use the ast templates it
failed, without that I don't think it is possible to write a (maintainable)
parser. The 1.7 release doesn't even compile, it gives a error in a certain
line.

DrX

Thanks, for your reply.

[Joel de Guzman]

"Moonlit" <al******@jupiter.universe> wrote in message
news:3f***********************@news.xs4all.nl

I just decided to order my copy of vc++ 2003, hope it works with spirit

and

well, I could always backtrack to flex++/byacc/bison++ if not.

The only problem I see with this is the latest version of VC++ still has
weak template support, and as you already know, Spirit is very template
intensive. Have you checked out the BOOST library, to which Spirit was
resently added too? It has some nice workarounds for various compilers.

Well they say over in the spirit newsgroup that vc7.1 compiles very well,
but of course, the proof of the...

I tried different versions of spirit 1.6.1 (the latest stable release).
Could create a simple parser but when I wanted to use the ast templates it
failed, without that I don't think it is possible to write a (maintainable)
parser. The 1.7 release doesn't even compile, it gives a error in a certain
line.

Hi,

AFAIK, 1.7 compiles cleanly on VC 7.1 (all regressions pass). Could you
be more specific with the compilation errors you are getting? I'd appreciate it
if you can post some code that exhibits the problem in Spirit's mailing list.
Rest assured, we are trying our best to make things work as smoothly as
possible.

Regards,
--
Joel de Guzman
http://www.boost-consulting.com
http://spirit.sf.net

[Moonlit]

Hi,

Thanks Joel four your reply and all your guys for working on spirit, I realy
love to just write C++ code and not have to mix C and/or precompile stuff.

I have got my VC7.1 version in. And it seems to compile fine (though I
didn't create any more than some simple things), but it did compile the same
example project that couldn't be compiled with VC6.0 and it works!

However with the VC6.0 the following errors are given as soon as I use ast
stuff (I could create a simple command line parser though) if you are
interested:

This is the ast_calc from the example section (the regular calc works fine
with VC6.0).

Compiling...
ast_calc.cpp
h:\include\boost\spirit\tree\common.hpp(207) : error C2984: 'node_iter_data'
: template parameters '' and '' do not match
h:\include\boost\spirit\tree\common.hpp(207) : see declaration of
'node_iter_data'
h:\include\boost\spirit\tree\common.hpp(534) : error C2989:
'tree_match_attr<class boost::reference_wrapper<T> >' : template class has
already been defined as a non-template class
h:\include\boost\spirit\tree\common.hpp(534) : error C2988: unrecognizable
template declaration/definition
h:\include\boost\spirit\tree\common.hpp(569) : fatal error C1903: unable to
recover from previous error(s); stopping compilation
Thanks for your great work,

Regards, Ron AF Greve

"Joel de Guzman" <dj****@gmx.co.uk> wrote in message
news:bm************@ID-58159.news.uni-berlin.de...

"Moonlit" <al******@jupiter.universe> wrote in message
news:3f***********************@news.xs4all.nl

I just decided to order my copy of vc++ 2003, hope it works with spirit and
well, I could always backtrack to flex++/byacc/bison++ if not.

The only problem I see with this is the latest version of VC++ still has weak template support, and as you already know, Spirit is very template
intensive. Have you checked out the BOOST library, to which Spirit was
resently added too? It has some nice workarounds for various

compilers.
Well they say over in the spirit newsgroup that vc7.1 compiles very well, but of course, the proof of the...

I tried different versions of spirit 1.6.1 (the latest stable release).
Could create a simple parser but when I wanted to use the ast templates it failed, without that I don't think it is possible to write a (maintainable) parser. The 1.7 release doesn't even compile, it gives a error in a certain line.
Hi,

AFAIK, 1.7 compiles cleanly on VC 7.1 (all regressions pass). Could you
be more specific with the compilation errors you are getting? I'd

appreciate it if you can post some code that exhibits the problem in Spirit's mailing list. Rest assured, we are trying our best to make things work as smoothly as
possible.

Regards,
--
Joel de Guzman
http://www.boost-consulting.com
http://spirit.sf.net