Lambda Expressions and Expression Trees

Hi there!

My name is Tim, and I’m the dev lead for the VB compiler team. Recently Amanda convinced me to blog on the VB team blog rather then my own blog for a variety of reasons, and so here I am. My current plan is that I’ll post VB related postings here, and more compiler implementation/technology/fun facts on my own blog (ie, things that don’t relate to the VB language/features).

Recently my article on lambda expressions got published on MSDN magazine. For this blog post, I’m going to expand on some of the ideas in that article, so feel free to check out the article if some of the ideas here don’t make sense yet.

Lambda expressions are really cool not only because of the reasons stated in the article, but also because it’s possible to capture the contents of a lambda expression in a readable form and treat it as data rather than executable code.

For example, the following line of code turns a lambda expression into a callable delegate:

Dim f As Func(Of Integer, Integer) = function(x) x * 2
Dim r = f(10)

.csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, “Courier New”, courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; }

In this code sample, the first line creates a lambda expression and does the magic to assign it to a callable delegate f, and the second line invokes f and returns 20.

This is all cool, and I hope my article highlights some of the reasons why I think this is cool. What’s even more cool is that you can actually assign a lambda expression into a special variable type and the compiler does some more magic for you:

Dim e As Expression(Of Func(Of Integer, Integer)) = function(x) x * 2

In this code sample, e is not a callable delegate. Rather, it’s an expression tree. An expression tree is a data representation of the lambda expression in a form that is easy for you to read and reason about the lambda expression in.

If you look through the value of e in the debugger, you’ll see that you can easily determine that the expression was a multiplication operation of variable called x of type integer with a constant integer 2.

By default, query operators that extend IEnumerable take lambda expressions. Query operators that extend IQueryable take expression trees. Therefore, when you issue a query over a type that is queryable, the compiler will generate an expression tree rather than a lambda expression.

Scott, one of the developers on my team, has written an excellent article as well on some of these ideas. He has a great diagram there showing the expression tree representation of a few lambda expressions.

What can you do with expression trees?

In general, you can do 2 things:

Reason about the data in the expression tree, and do custom processing over it. You can also convert the data in the expression tree into another domain (such as XML).
Use the .Compile() method to turn the expression tree into a delegate so that you can execute it.
Build queries with dynamic conditions: see Jonathan’s (our PM) excellent post. He has a great example of how you can manipulate the expression trees and build custom conditions in your lambda expressions.

Item 2 is interesting, because it means that you can write a lambda expression into an expression tree, reason about it, etc, and then turn it into a callable delegate.

Dim e As Expression(Of Func(Of Integer, Integer)) = function(x) x * 2
Dim f = e.Compile
Dim r = f(10)

In this example, the value of r is 20, just like the previous example where we assigned the lambda directly to a delegate type. .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, “Courier New”, courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; }

Why is this interesting?

Without expression trees, its impossible to describe expressions in a descriptive format; the only description of them is IL, which is too low level to re-create the user’s intent for the expression. Rather, expression trees provide a high level tree representation of an expression, so that you can easily understand the user’s intent for the expression.

Here are some uses of expression trees that may be interesting:

Reason about expressions, then convert them to delegates and call them.
Convert expressions into SQL so that they can be executed by a server (this is what LINQ to SQL does).
Convert expressions into XML and write them to disk.
Convert expressions into a custom format to send over a network protocol to a server that can then rebuild the expression tree from the format, recreate the expression, and reason about it and/or execute it.

Over the next several posts, I’m going to blog specifically about the expression trees that the VB compiler generates, and some differences between C# trees and VB trees in order to maintain VB semantics. I will also provide tips on things you can do to be sure that any library that you write that consumes expression trees will provide the best experience for VB developers.

Tim