Expression Builder

An object that provides a fluent interface over a normal command-query API

APIs are usually designed to provide a set of self standing methods on objects, ideally these methods can be understood individually. I call this style of API a command-query API, it's so normal that we don't have a general name for it. DSLs require a different kind of API, what I call a fluent interface, which is designed around the readability of a whole expression. Fluent interfaces lead to methods that make little sense individually, and often violate the rules for good command-query APIs.

An Expression Builder provides a fluent interface as a separate layer on top of the regular API. This way you have both styles of interface and the fluent interface is clearly isolated, making it easier to follow.

How it Works

A Expression Builder is an object, or group of objects that provides a fluent interface which it then translates into calls on an underlying command-query API. You can think of it as a translation layer that translates the fluent interface into the command-query API.

Exactly how you arrange Expression Builders depends very much of the kind of clause you are dealing with. Method Chaining is a sequence of method calls with each one returning an Expression Builder, Nested Function may use an Expression Builder that is a superclass or set of global functions. As a result I can't really give any general rules for what an Expression Builder looks like in this pattern - you need to look at the different kinds of Expression Builder shown in the other internal DSL patterns. What I can do, is talk a little about some general guidelines that I think will help to put together a clear layer of Expression Builders.

In simpler cases you may just a single Expression Builder for your DSL, in more complex cases you may have multiple Expression Builders handling different parts of the language. These Expression Builders may come in different forms as they combine together.

One of the most useful tips to getting a clear separated set of Expression Builders is to ensure you have a clear Semantic Model. The Semantic Model should have objects with command-query interfaces and can be manipulated without using any fluent constructs. You can verify this by being able to write tests for the Semantic Model that don't use any DSLs as part of it. It may not be wise to force this rule too much. After all the whole point of an internal DSL is to make it easier to work with these objects, so usually it will be easier to manipulate them in tests with the DSL than with the command-query interface. But I'd usually include at least some tests that only use the command-query interface.

Expression Builders then can act on top of these model objects. You should be able to test the Expression Builders by comparing the Semantic Model objects they manipulate with direct calls to the Semantic Model command-query APIs.

When to use it

I find Expression Builder a default pattern - meaning I tend to use it pretty much all the time unless there's a good reason not to.

This, of course, begs the question of when are there occasions when Expression Builder isn't a good idea?

The main case I've run into is when the Semantic Model itself is designed to be entirely used in a fluent way. In this case there's no need for a translation layer. However it does mean that everyone who uses the Semantic Model must be comfortable with a fluent way of working and you don't have cases where a command-query API is used as well. The danger really is in mingling command-query and fluent methods on the same object - the result is just too confusing.

A fluent Semantic Model is a rare case because fluent interfaces are themselves rare, and thus more likely to cause confusion for programmers unfamiliar with them. Hence my preference to create a normal Semantic Model and layer the fluency over it. However if you think your team is comfortable with fluency all the way down, then that's an approach to consider.

Example: JMock (Java)

Since Expression Builder takes different forms depending on the particular type of fluent construct you are using, I can't really write a simple example of Expression Builder. So instead my example will show how you combine multiple simple Expression Builders. As a result this example won't fit your reading flow if you are reading this section for the first time in a sequential format. If you are in that state I'd suggest getting comfortable with some of the later patterns first - in particular Method Chaining and Object Scoping.

JMock, a Java mock object library written by Steve Freeman and Nat Pryce, has been a significant influence on my thinking on internal DSLs. They really pushed the boundaries of fluency in a mainstream static language. Here I'll look at the way they different forms Expression Builder to process their language. The example is based on the original JMock, as it's the one that I found particularly interesting. JMock has moved on since, but I think the ideas here are worth looking at. I stole the example from Freeman and Pryceand used the code from the JMock CVS repository tagged NV1_1_0.

In their own work, they don't use the term Expression Builder or indeed most of the other vocabulary I use in this book (it would be impossible since they came first). Rather they talk about having two layers in JMock - a syntax (they also say "builder") layer and an interpreter layer. The interpreter layer is the Semantic Model and the syntax layer is a layer of Expression Builders. (I've gone back and forth on whether the pattern should be Expression Builder or "builder layer". It's the same basic idea and the different formulations of the pattern (object or layer) work better in different contexts.)

Mock objects are a form of Test Double where you provide a mock collaborator to an object under test. The mock records all the method invocations that the object under test invokes against it and verifies that these invocations match a pre-defined set of expectations. The DSL I'm looking at here focuses on how you specify these expectations.

I'll use a single example as the focus for looking at how this all works. Imagine a system that automates trading. This agent needs to buy some stock when a price reaches a certain threshold. As part of this trade the agent needs to call a mainframe interface's buy method with a set quantity and call a auditing system giving it a ticket which it got from the return value of the mainframe interface. The expectation can be encoded like this:

Mock mainframe = mock(Mainframe.class);
Mock auditing = mock(Auditing.class);
Agent agent =
  new Agent( QUANTITY,
             (Mainframe)mainframe.proxy(),
             (Auditing)auditing.proxy() );


public void testBuysWhenPriceEqualsThreshold() {
  mainframe.expects(once())
    .method("buy").with(eq(QUANTITY))
    .will(returnValue(TICKET));
  auditing.expects(once())
    .method("bought").with(same(TICKET));
  agent.onPriceChange(THRESHOLD);
}

The first three statements here show some variables that need to set up: mock objects for the mainframe and auditing objects and an instance of the agent which refers to these mocks.

The method testBuysWhenPriceEqualsThreshold is the test case method. Like most mock-style test cases it comes in two parts. The first two statements set expectations on the mainframe and auditing mocks. The final statement tells the agent to act - machinery within the Semantic Model will then verify that the mocks received the right messages. We'll ignore that machinery here and just concentrate on those two expectations, which are defined using the JMock expectation DSL.

Looking more at expectation definition, we can see Method Chaining: method, with, and will. As well as Method Chaining, however there's also Nested Functions: once, eq, and will. These Nested Functions are defined through Object Scoping. Any tests using JMock must be written in a subclass of MockObjectTestCase, which provides methods that will work in Nested Functions.

The expectation language is triggered by the call to expects.

The expects method is defined on the mock object

class Mock...
    public NameMatchBuilder expects( InvocationMatcher expectation ) {
        NameMatchBuilder builder = addNewInvocationMocker();
        builder.match(expectation);
        return builder;
    }
    private NameMatchBuilder addNewInvocationMocker() {
        InvocationMocker mocker = new InvocationMocker(new InvocationMockerDescriber());
        addInvokable(mocker);
        return new InvocationMockerBuilder( mocker, this, getMockedType() );
    }

class InvocationMockerBuilder...
    public MatchBuilder match( InvocationMatcher customMatcher ) {
        return addMatcher(customMatcher);
    }
    private InvocationMockerBuilder addMatcher( InvocationMatcher matcher ) {
        mocker.addMatcher(matcher);
        return this;
    }

The expects method and subroutines create a new Semantic Model object (the InvocationMocker), wraps it in a builder (the InvocationMockerBuilder), adds a matcher to the Semantic Model and returns the builder for chaining.

There's an interesting point in what expects actually returns. Although the code creates an instance of InvocationMockerBuilder, the return type is NameMatchBuilder. This is a use of progressive interfaces in Method Chaining - each step in the chain returns the same object but wrapped in a different interface. The reason for this is to guide the user through IDE's completion. At each step in the chain an IDE's method completion pop-up will only show those methods that are legal at that point in the expression. This makes the DSL much easier to use.

JMock uses "builder" as part of method chaining, but the MockObjectTestCase that supports Object Scoping is also part of the syntax layer. So in the way I'm looking at things both InvocationMockerBuilder and MockObjectTestCase are Expression Builders as they both carry out the translation from DSL script to the underlying Semantic Model.

The use of both Method Chaining and Nested Function with Object Scoping raised another interesting point about the design of the DSL. One of the results of mixing the two was that it allowed support for appropriate extension of the DSL. With Method Chaining the choice of terms you can use is fixed by the DSL framework - essentially by the methods defined on InvocationMockerBuilder. However the methods used in Nested Function allow more extension. As well as using the methods defined on MockObjectTestCase you can also use any methods that you define in the particular test class itself. This makes it easy to add new clauses to the DSL. A good example of this is the argument matching system.

The method method returns the InvocationMockerBuilder with the interface ArgumentsMatchBuilder.

public interface ArgumentsMatchBuilder extends MatchBuilder {
    MatchBuilder with( Constraint[] argumentConstraints );
    MatchBuilder with( Constraint arg1 );
    MatchBuilder with( Constraint arg1, Constraint arg2 );
    MatchBuilder with( Constraint arg1, Constraint arg2, Constraint arg3 );
    MatchBuilder with( Constraint arg1, Constraint arg2, Constraint arg3, Constraint arg4 );
    MatchBuilder withNoArguments();
    MatchBuilder withAnyArguments();
}

    public MatchBuilder with( Constraint arg1 ) {
        return with(new Constraint[]{arg1});
    }
    public MatchBuilder with( Constraint[] constraints ) {
        return addMatcher(new ArgumentsMatcher(constraints));
    }

The argument to with is one or more constraints. Through Object Scoping you a couple of dozen built in constraints - one of which is eq as seen here.

class MockObjectSupportTestCase...
    public IsEqual eq( Object operand ) {
        return new IsEqual(operand);
    }

Others include such things as IsCollectionContaining, IsNothing, IsGreaterThan; even Or and And. (MockObjectSupportTestCase is a parent of MockObjectTestCase) People who have run into it will recognize the Specification pattern in play, with its characteristic use of composite logic terms and leaf predicates. Using specification here allows you to build a very wide range of possible conditions on method parameters.

Using specification with Object Scoping gives you a further flexibility - you can define your own constraint methods in the test class. After all any method in the concrete test class can be used inside the Nested Function.

The limitation that Method Chaining fixes the method calls goes away if you have a mechanism to add extenstion method to the builder - as in C#'s partial methods or Ruby's open classes. This option isn't available in this case, but increases the option in other languages.

This use of constraints is another are where JMock has proved very influential. Many of the XUnit testing frameworks have added constraint based matching to their assertion syntax. It doesn't make code that much more readable, but it does give more information to the framework which allows the framework to provide more descriptive error messages.

So going back to the distinction between Method Chaining and Object Scoping. Method Chaining fixes the methods you can use in the DSL, but Object Scoping allows you to easily add more methods. So one reason for choosing Object Scoping is in places where the framework designers wanted that easy extensibility. As usual with any system the designers have to choose what is is easy to extend and what is hard. Trying to make everything easy to extend results in too little structure which only makes everything much harder to use.