Linq; expression parser?

[Marc Gravell]

In Linq, you can apparently get a meaningful body from and
expression's .ToString(); random question - does anybody know if linq
also includes a parser? It just seemed it might be a handy way to
write a safe but easy implementation (i.e. no codedom) for an
IBindingListVie w.Filter (by compiling to a Predicate<T>).

Anybody know if this is possible at all?

Marc

[Frans Bouma [C# MVP]]

Marc Gravell wrote:

In Linq, you can apparently get a meaningful body from and
expression's .ToString(); random question - does anybody know if linq
also includes a parser? It just seemed it might be a handy way to
write a safe but easy implementation (i.e. no codedom) for an
IBindingListVie w.Filter (by compiling to a Predicate<T>).

Anybody know if this is possible at all?

Why would you use a parser on the string output? Because that parser
will produce a parse tree which will look very similar to the
expression tree you called ToString() on :)

So interpret the expression tree instead.

FB

--
------------------------------------------------------------------------
Lead developer of LLBLGen Pro, the productive O/R mapper for .NET
LLBLGen Pro website: http://www.llblgen.com
My .NET blog: http://weblogs.asp.net/fbouma
Microsoft MVP (C#)
------------------------------------------------------------------------

[Jon Skeet [C# MVP]]

Frans Bouma [C# MVP] <pe************ ******@xs4all.n lwrote:

Marc Gravell wrote:

In Linq, you can apparently get a meaningful body from and
expression's .ToString(); random question - does anybody know if linq
also includes a parser? It just seemed it might be a handy way to
write a safe but easy implementation (i.e. no codedom) for an
IBindingListVie w.Filter (by compiling to a Predicate<T>).

Anybody know if this is possible at all?

Why would you use a parser on the string output? Because that parser
will produce a parse tree which will look very similar to the
expression tree you called ToString() on :)

So interpret the expression tree instead.

I think the point would be to parse expression trees which *hadn't*
been created from ToString, but read in from a file etc. I understood
Marc's introductory sentence to effectively mean: "there's a useful
format for LINQ expressions, as shown by the ToString method".

--
Jon Skeet - <sk***@pobox.co m>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[Marc Gravell]

Jon has correctly interpreted my witterings... my point being, that
unlike raw .Net 2.0 predicates (etc), LINQ expression trees are quite
structured. Obviously the C# 3-series compiler can parse source to
create an expression, but it is unclear to me (at least until
Reflector supports CLR 3.5 ;-p) how much of this is the compiler and
how much is the runtime.

The ability to parse a string to an expression would be very
powerful... although there would obviously be some limitations in
terms if resolving external entities... but heck: just access to the
expression arguments and literals would be pretty powerful.

Maybe I'm just "off on one"...

Marc

[Frans Bouma [C# MVP]]

Jon Skeet [C# MVP] wrote:

Frans Bouma [C# MVP] <pe************ ******@xs4all.n lwrote:

Marc Gravell wrote:

In Linq, you can apparently get a meaningful body from and
expression's .ToString(); random question - does anybody know if
linq also includes a parser? It just seemed it might be a handy
way to write a safe but easy implementation (i.e. no codedom) for
an IBindingListVie w.Filter (by compiling to a Predicate<T>).
>
Anybody know if this is possible at all?

Why would you use a parser on the string output? Because that
parser will produce a parse tree which will look very similar to the
expression tree you called ToString() on :)

So interpret the expression tree instead.

I think the point would be to parse expression trees which hadn't
been created from ToString, but read in from a file etc. I understood
Marc's introductory sentence to effectively mean: "there's a useful
format for LINQ expressions, as shown by the ToString method".

I 'm not sure if I understand you correctly. The C# sourcecode is
parsed to code which builds an expression tree (if I understood
everything correctly) which at runtime results in an Expression tree
with objects. This tree is then passed to the object which is the
source for the query, for example the o/r mapper engine which will
convert the tree to a sql query.

Expression trees aren't serializable, so to store them you need a
representation, which could be a string indeed. However I don't see a
use case for that, as the expression tree only lives at runtime anyway.

Now, to parse the string, you WILL end up with a parse tree (or your
parser isn't that maintainable ;)). This parse tree has likely a lot in
common with the expression tree, so you have to write the tree
traversal code / node interpretation code anyway. So why write the
parser as well, as you already get the expression tree handed to you?

FB

--
------------------------------------------------------------------------
Lead developer of LLBLGen Pro, the productive O/R mapper for .NET
LLBLGen Pro website: http://www.llblgen.com
My .NET blog: http://weblogs.asp.net/fbouma
Microsoft MVP (C#)
------------------------------------------------------------------------

[Frans Bouma [C# MVP]]

Marc Gravell wrote:

Jon has correctly interpreted my witterings... my point being, that
unlike raw .Net 2.0 predicates (etc), LINQ expression trees are quite
structured. Obviously the C# 3-series compiler can parse source to
create an expression, but it is unclear to me (at least until
Reflector supports CLR 3.5 ;-p) how much of this is the compiler and
how much is the runtime.

it's my understanding that there are two routes:
1) the source the query will work on implements IEnumerable (e.g.
List<T>) This will make the compiler emit calls to extension methods.
2) the source the query will work on implements IQueryable. This will
make the compiler emit code which builds an expression tree. (correct
me if I'm wrong, this is what I understood of it).

So, the code executed at runtime, will build the expression tree for
you and hand it to the source object the query works on (in situation
2). From there, the source object is on its own and can do with the
expression tree what it wants.

This also means that errors in the tree could lead to
runtime-exceptions, not compile-time exceptions. For example the
reference to a method which isn't available according to the
sourceobject.

Now, what it also means is that you don't need a string parser, as
that would lead to a tree similar to the expression tree and you then
have to write the tree intepreter as well, so you win nothing from
using the string parser. There's one exception: expression trees aren't
serializable IIRC, so to serialize them, you could opt for text,
however references to properties/methods are hard to re-build I think.

The ability to parse a string to an expression would be very
powerful... although there would obviously be some limitations in
terms if resolving external entities... but heck: just access to the
expression arguments and literals would be pretty powerful.

Parse a string to an expression is what every parser does ;). You
already have that build into the C# compiler (v3.5), so why re-do that?

FB

Maybe I'm just "off on one"...

--
------------------------------------------------------------------------
Lead developer of LLBLGen Pro, the productive O/R mapper for .NET
LLBLGen Pro website: http://www.llblgen.com
My .NET blog: http://weblogs.asp.net/fbouma
Microsoft MVP (C#)
------------------------------------------------------------------------

[Jon Skeet [C# MVP]]

Frans Bouma [C# MVP] <pe************ ******@xs4all.n lwrote:

<snip>

The ability to parse a string to an expression would be very
powerful... although there would obviously be some limitations in
terms if resolving external entities... but heck: just access to the
expression arguments and literals would be pretty powerful.

Parse a string to an expression is what every parser does ;). You
already have that build into the C# compiler (v3.5), so why re-do that?

Because you might want to do it at execution time. Suppose the queries
are stored in a configuration file - you don't really want to have to
build valid C# to give it to the C# compiler, if there's a parser which
deals specifically with LINQ queries.

--
Jon Skeet - <sk***@pobox.co m>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[Marc Gravell]

Maybe I'm not explaining myself well...
My point being; the creation of expression trees is not solely the
domain of the compiler (you can create them at runtime through code)
[equally, neither is their execution, hence D-LINQ, object-LINQ, etc).

Occasionally in a system it is necessary to determine some factors at
runtime, for instance through configuration. Rather than write yet
another parser, it would seem highly advantageous if LINQ allowed us a
common supported expression parser. I can think of many uses of such.

It doesn't sound as though anybody is aware of anything... maybe I'll
see what I can cobble together with expression trees. Pity.

Marc

[Frans Bouma [C# MVP]]

Jon Skeet [C# MVP] wrote:

Frans Bouma [C# MVP] <pe************ ******@xs4all.n lwrote:

<snip>

The ability to parse a string to an expression would be very
powerful... although there would obviously be some limitations in
terms if resolving external entities... but heck: just access to
the expression arguments and literals would be pretty powerful.

Parse a string to an expression is what every parser does ;). You
already have that build into the C# compiler (v3.5), so why re-do
that?

Because you might want to do it at execution time. Suppose the
queries are stored in a configuration file - you don't really want to
have to build valid C# to give it to the C# compiler, if there's a
parser which deals specifically with LINQ queries.

If the queries are stored in a config file, they're not Linq queries,
IMHO. So if the proposal is: <text in some format-Expression trees,
sure, that can be helpful, but only if the expression tree is then
usefully interpreted by the interpreter you want to use.

The expression tree interpreter and the actual class/object which
executes the interpretation result are very tightly connected. This
means that you can't have an expression tree parser without a class
which does something with the interpretation of the tree. The tree
isn't the goal, it's an intermediate format between Linq query in VB/C#
and a series of commands to execute.

IMHO, it's only useful to store expression trees as text somewhere if
you can't have the C# code somewhere AND the series of commands you
want to execute is produced by a class which only eats expression
trees. Any other situation doesn't justify the route of expression
trees IMHO.

FB

--
------------------------------------------------------------------------
Lead developer of LLBLGen Pro, the productive O/R mapper for .NET
LLBLGen Pro website: http://www.llblgen.com
My .NET blog: http://weblogs.asp.net/fbouma
Microsoft MVP (C#)
------------------------------------------------------------------------

[Frans Bouma [C# MVP]]

Marc Gravell wrote:

Maybe I'm not explaining myself well...
My point being; the creation of expression trees is not solely the
domain of the compiler (you can create them at runtime through code)
[equally, neither is their execution, hence D-LINQ, object-LINQ, etc).

Occasionally in a system it is necessary to determine some factors at
runtime, for instance through configuration. Rather than write yet
another parser, it would seem highly advantageous if LINQ allowed us a
common supported expression parser. I can think of many uses of such.

What's needed is solid documentation how an expression tree looks
like when given Linq elements are expressed in C#/VB. A generic
expression parser is actually not that useful. All it can do is tell
you what kind of node is found at a given position: it doesn't do
anything for you. The tough element is to DO something useful AFTER a
given node or treebranch has been examined and understood.

Interpreting a string similar to interpreting an expression tree is
equal to interpreting the expression tree. At least if you're using an
LL(n) parser. An LR(n) parser shifts/reduces the parse-tree while
evaluating it AND building it so you can best describe the expression
tree as a parse result of an LL(n) parse operation of input text. The
tree is then handed over to the engine which consumes the tree for
doing something with it, be it emitting code, constructing commands etc.

Though, you only need to go that route if you work with the end part
of the sequence just described, i.e. the engine which consumes an
expressiontree. Then you need to feed it an expression tree to produce
something.

If you then need to build queries at runtime, you indeed are out of
luck UNLESS you build the expression tree manually.

That's also why I don't understand why you want to use strings which
are outputted from the expression tree in the first place: you already
have the tree! Not only that, going from strings to expression tree
with a parser effectively re-implements what's already in the C#
compiler. I really fail to see why the effort is needed or even
'powerful' or useful. Please give me a use-case scenario, as things
from configuration is too vague: what are you going to do with the data
from the config file? Why is the strings -expression tree the only
real solution to your situation?

Besides that, generating a piece of C# with the linq queries, compile
it and run it at runtime takes 100 lines of code max.

It doesn't sound as though anybody is aware of anything... maybe I'll
see what I can cobble together with expression trees. Pity.

There are some blank spots here and there, but the overall system
isn't that obscure.

What bugs me is that I still have no clue what you're trying to
achieve.

FB

--
------------------------------------------------------------------------
Lead developer of LLBLGen Pro, the productive O/R mapper for .NET
LLBLGen Pro website: http://www.llblgen.com
My .NET blog: http://weblogs.asp.net/fbouma
Microsoft MVP (C#)
------------------------------------------------------------------------