Nested Closure

Express statement sub-elements of a function call by putting them into a closure in an argument.

computer do 
  processor do 
    cores 2
    i386
    speed 2.2
  end
  disk do
    size 150
  end
  disk do
    size 75
    speed 7200
    sata
  end
end

How it Works

The basic idea of a Nested Closure is similar to that of Nested Function, but the child expressions of the function call are wrapped in a closure. To show the difference, here's a call to create a new processor using Nested Function in Ruby:

processor(
  cores 2,
  i386
)

Now with a Nested Closure:

processor do
  cores 2
  i386
end

Instead of passing two Nested Function arguments I pass a single Nested Closure argument which contains the two Nested Functions. (I'm using Ruby here as it provides closures in a syntax which is suitable for this discussion.)

Placing the sub-elements in a Nested Closure has an immediate consequence for my implementation - I have to put in code to evaluate the closure. With a Nested Function I don't need to do this since the language automatically evaluates the cores and i386 functions before calling the processor function. With a closure argument the processor function is called first and the closure is only evaluated when I explicitly program it to. So usually I'll evaluate the closure within the body of the processor function. The processor function can also carry out other tasks before and after the closure evaluation, such as setting up Context Variables.

In the example above the contents of the closure is a Function Sequence. One of the problems of a Function Sequence is that the multiple functions communicate using hidden Context Variables. While you still have to do this inside a Nested Closure, the processor function can create the Context Variable before evaluating the closure and tear it down afterwards. This can greatly reduce the problem of lots of Context Variables appearing all over the place.

Another choice for the sub-elements is to use Method Chaining. Here there is the additional benefit that the parent function can set up the head of the chain and pass it into the closure as an argument.

processor do |p|
  p.cores(2).i386
end

It's also quite common to pass in a Context Variable as an argument.

processor do |p|
  p.cores 2
  p.i386
end

In this case we have a Function Sequence but with the Context Variable explicitly present. This often makes it easier to follow, without adding too much clutter.

Bare functions written inside a Nested Closure are evaluated in the scope where they are defined, so again it's usually wise to use Object Scoping. Passing in an explicitly Context Variable or using Method Chaining allows you to avoid this, as well as to organize the builder code into different builders.

Some languages allow you to manipulate the context that a closure is executed in. This can allow you to use bare functions and still use multiple builders. The example with Ruby's instance_eval shows how this can work.

In the examples I've shown above, I've put all the sub-elements of the parent function into a single closure. It's also possible to use multiple closures. The advantage of this is that it allows you to evaluate each sub-closure independently. A good example of where this is handy is where you have a conditional expression, such as this example in Smalltalk:

aRoom 
  ifDark: [aLight on] 
  ifLight: [aLight off]

When to use it

Nested Closure is a useful technique because it combines the explicitly hierarchic structure of Nested Function with the ability to control when the arguments are evaluated. Control of evaluation provides you with a lot of flexibility helping you to avoid many of the limitations of Nested Function.

The biggest limitation of Nested Closure is the way the host language supports closures. Many languages don't provide closures at all. Those that do often provide the syntax in a way that doesn't jive terribly well with DSLs, such as an awkward keyword.

It's usually worth thinking of Nested Closure as a enhancement to Nested Function, Function Sequence, and Method Chaining. The explicit control of evaluation gives you different advantages with each technique. All of these, however, boil down to the fact that you can do specific setup and tear-down operations on either side of the closure invocations. With Function Sequence this means you can prepare Context Variables right before they are used by the closure. With Method Chaining you can setup the head of the chain before invoking the closure.

Example: Wrapping a Function Sequence in a Nested Closure (Ruby)

For a first example I'll start with what I might consider as the most straightforward case, using a Nested Closure in conjunction with a Function Sequence. Here is the the DSL script

class BasicComputerBuilder < ComputerBuilder
  def doBuild
    computer do 
      processor do 
        cores 2
        i386
        processorSpeed 2.2
      end
      disk do
        size 150
      end
      disk do
        size 75
        diskSpeed 7200
        sata
      end
    end
  end
end

To begin the discussion, let's compare this with a version using Function Sequence alone, which would look like this

class BasicComputerBuilder < ComputerBuilder
  def doBuild
    computer
      processor 
        cores 2
        i386
        processorSpeed 2.2
      disk
        size 150
      disk 
        size 75
        diskSpeed 7200
        sata
  end
end

From the script's point of view the only change with Nested Closure is to add the do-end closure delimiters. By adding these, I add an explicit hierarchic structure to what otherwise is a linear sequence with a formatting convention. The extra syntax doesn't strike me as troubling because it's marking the structure from the reader's point of view in a way that makes sense to the reader.

Now lets move onto the implementation. As usual I'm using Object Scoping so I can have bare functions resolve against an Expression Builder. (A note for rubyists: I'm using subtyping here for pedagogical reasons, but with ruby I'd usually use instance_eval.) We can see the basic structure of using Nested Closure by looking at the clause for computer.

class ComputerBuilder...
  def computer &block
    @result = Computer.new
    block.call
  end

I pass in the closure as an argument (Ruby refers to closures as "blocks"), set up some context and then call the closure. The processor function can then use this context and repeat the process for its children.

class ComputerBuilder...
  def processor &block
    @result.processor = Processor.new
    block.call
  end
  def cores arg
    @result.processor.cores = arg
  end
  def i386
    @result.processor.type = :i386
  end
  def processorSpeed arg
    @result.processor.speed = arg
  end  

I do the same for the disks. The only difference is that this time I use the more idiomatic yield keyword to call the implicitly passed in block. (This is a mechanism ruby uses to simplify working with a single block argument.)

class ComputerBuilder...
  def disk
    @result.disks << Disk.new
    yield
  end
  def size arg
    @result.disks.last.size = arg
  end
  def sata
    @result.disks.last.interface = :sata
  end
  def diskSpeed arg
    @result.disks.last.speed = arg
  end

Example: Simple C# example (C#)

For contrast, here is pretty much the same example using C# instead.

  class Script : Builder {
    protected override void doBuild() {
      computer(() => {
        processor(() => {
          cores(2);
          i386();
          processorSpeed(2.2);
        });
        disk(() => {
          size(150);
        });
        disk(() => {
          size(75);
          diskSpeed(7200);
          sata();
        });
      });
    }
  }

As you can see the structure is exactly the same as the Ruby example, the big difference is that there's a lot more punctuation in the script.

The builder also looks remarkably similar

class Builder...
    protected void computer(BuilderElement child) {
      result = new Computer();
      child.Invoke();
    }
    public delegate void BuilderElement();
    private Computer result;

The computer function follows the same pattern that we see in the Ruby case: pass the closure argument, do any setup, invoke the closure, then do any tear-down. The most notable difference here with C# is that we have to define the type of the closure we pass in with a delegate clause. In this case the closure has no arguments and no return type, but with a more complicated case we might need several different types.

The rest of the code is similarly similar to the Ruby case, so I'll save the ink.

To my eyes, Nested Closure works much less well here in C# than it did in Ruby. Ruby's do...end closure delimiters flow more naturally to me than C#'s ()=> {...}, particularly when you also add the mandatory parenthesis into the mix. (You can also use {...} as closure delimiters in Ruby.) The more used you are to C# notation, the less that will bother you. Furthermore this example doesn't pass arguments into the closure - which adds more punctuation to the Ruby case but actually makes the C# easier to read since the empty parentheses now have something to surround.

Example: Using Method Chaining (Ruby)

Nested Closure can work in a couple of different styles. Here's an example using Method Chaining.

ComputerBuilder.build do |c|
  c.
    processor do |p|
      p.cores(2).
        i386.
        speed(2.2)
    end.
    disk do |d|
      d.size 150
    end.
    disk do |d|
      d.size(75).
        speed(7200).
        sata
    end
end

The difference here is that each call passes in an object to the closure that's used at the head of a chain. The use of closure arguments arguably adds noise to the DSL script (as does the need to now wrap method arguments in parentheses), but one benefit is that you no longer need Object Scoping and thus can easily use the code in a fragmentary style.

Invoking the build method creates an instance of the builder and passes it into the closure as an argument.

class ComputerBuilder...
  attr_reader :content
  def initialize
    @content = Computer.new
  end
  def self.build &block
    builder = self.new
    block.call(builder)
    return builder.content
  end

Another useful part of this approach is that it makes it easy to factor the various builder methods into a group of small, cohesive Expression Builders. The processor clause introduces a new builder (using the more compact yield keyword).

class ComputerBuilder...
  def processor 
    p = ProcessorBuilder.new
    yield p
    @content.processor = p.content
    return self
  end

class ProcessorBuilder
  attr_reader :content
  def initialize
    @content = Processor.new
  end
  def cores arg
    @content.cores = arg
    self
  end
  def i386
    @content.type = :i386
    self
  end
  def speed arg
    @content.speed = arg
    self
  end  
end

Disks are also handled with a disk builder.

class ComputerBuilder...
  def disk 
    currentDisk = DiskBuilder.new
    yield currentDisk
    @content.disks << currentDisk.content
    return self
  end

class DiskBuilder
  attr_reader :content
  def initialize
    @content = Disk.new
  end
  def size arg
    @content.size = arg
    self
  end
  def sata
    @content.interface = :sata
    self
  end
  def speed arg
    @content.speed = arg
    self
  end
end

As well as allowing better factoring of the builder methods, it also allows me to use an unqualified "speed" method for both the processor and the disk without ambiguity.

Example: Function Sequence with explicit Closure arguments (Ruby)

In the previous example we saw there are several advantages to breaking down the language layer into multiple Expression Builders. With this approach each builder is smaller and more cohesive, we also allow clauses in different parts of the language to use the same name (as in processor and disk speed). The explicit closure arguments also allow us to easily use the DSL in a fragmentary context.

While Method Chaining gives us these advantages, the resulting DSL script can look rather awkward. The interplay between Nested Closure and Method Chaining doesn't necessarily fit will. Certainly most of the Ruby DSLs I've looked at do not use this style.

Instead the Ruby DSLs I've looked at use Function Sequence within each closure but pass an explicit closure argument to allow multiple builders. In this style our computer configuration script looks like this:

ComputerBuilder.build do |c|
  c.processor do |p|
    p.cores 2
    p.i386
    p.speed 2.2
  end
  c.disk do |d|
    d.size 150
  end
  c.disk do |d|
    d.size 75
    d.speed 7200
    d.sata
  end
end

The big difference here in the DSL script is that you have separate statements for each clause in the DSL. On every statement you have to state the passed in object as the receiver of the method call. Although this adds more text to the statement, it results in a more regular style of code that rubyists find easier to work with.

The implementation is very similar to the Method Chaining case. Again I have a computer builder at the top level with a class method that creates an instance and passes it to the supplied closure.

class ComputerBuilder...
  attr_reader :content
  def initialize
    @content = Computer.new
  end
  def self.build 
    builder = self.new
    yield builder
    return builder.content
  end

The processor clause introduces a new builder

class ComputerBuilder...
  def processor &block
    p = ProcessorBuilder.new
    yield p
    @content.processor = p.content
  end

class ProcessorBuilder
  attr_reader :content
  def initialize
    @content = Processor.new
  end
  def cores arg
    @content.cores = arg
  end
  def i386
    @content.type = :i386
  end
  def speed arg
    @content.speed = arg
  end  
end

I'll leave you the barely existing challenge of figuring out what the disks look like.

Example: Using Ruby's instance_eval (Ruby)

Passing in explicit closure arguments yield many advantages, but at the cost of constantly mentioning the name the of the argument. Ruby gives us a particularly nifty technique to help deal with this: the instance_eval method.

When you call a ruby block the block is evaluated in the context of where it's defined. In particular any bare functions (or fields) are resolved to the object in which it's defined. Using you can change this, telling some other object to execute the block within its context, which means any bare methods now resolve to the new object. The following code demonstrates the difference.

class StaticContext < Test::Unit::TestCase
  def identify
    return "in static context"
  end
  def test_demo
    o = OtherObject.new
    assert_equal "in static context", o.use_call {identify}
    assert_equal "in other object", o.use_instance_eval {identify}
  end
end

class OtherObject
  def identify 
    return "in other object"
  end  
  def use_call &arg
    arg.call
  end
  def use_instance_eval &arg
    instance_eval &arg
  end
end

In effect using instance_eval changes what self refers to inside the passed in block.

We can use this facility to allow us to use multiple builders with bare method calls in our DSL script.

ComputerBuilder.build do 
  processor do 
    cores 2
    i386
    speed 2.2
  end
  disk do
    size 150
  end
  disk do
    size 75
    speed 7200
    sata
  end
end

The builder takes the block as it did before, but uses instance_eval rather than call.

class ComputerBuilder...
  def self.build &block
    builder = self.new
    builder.instance_eval &block
    return builder.content
  end
  def initialize
    @content = Computer.new
  end    

Handling the processor again uses instance_eval.

class ComputerBuilder...
  def processor &block
    @content.processor = ProcessorBuilder.new.build(block)
  end

class ProcessorBuilder
  def build block
    @content = Processor.new
    instance_eval(&block)
    return @content
  end
  def cores arg
    @content.cores = arg
  end
  def i386
    @content.type = :i386
  end
  def speed arg
    @content.speed = arg
  end  
end

As does the disk.

class ComputerBuilder...
  def disk &block
    @content.disks << DiskBuilder.new.build(block)
  end

class DiskBuilder
  def build block
    @content = Disk.new
    instance_eval(&block)
    return @content
  end
  def size arg
    @content.size = arg
  end
  def sata
    @content.interface = :sata
  end
  def speed arg
    @content.speed = arg
  end

The way I've shown the DSL script use is typical for a fragmentary context where I'm putting a little bit of DSL into a regular ruby program. In a self contained context I can have the DSL script in its own file. In which case by using instance_eval I don't need any top and tail noise to set up Object Scoping - the whole script file looks like this.

computer do 
  processor do 
    cores 2
    i386
    speed 2.2
  end
  disk do
    size 150
  end
  disk do
    size 75
    speed 7200
    sata
  end
end

The builder can then process the whole file by instance_evaling it.

class ComputerBuilder...
  def load aStream
    instance_eval aStream
  end
  def computer
    yield
  end

Using instance_eval seems such a good trick that you can wonder if you should ever pass explicit closure arguments. As it turns out there is a very real choice, one that was crystallized for me with Jim Weirich's experience with his builder library. The builder library is a very nice library to create XML documents using Nested Closures and Dynamic Reception. In the first version of the library Jim used instance_eval, but later switched to explicit parameters. The reason is that programmers are used to the call behavior with closures - redefining self causes a lot of confusion as well as making it very difficult to refer to elements in the static context that you need.

For me the choice lies in whether you are using the DSL script in a self-contained or fragmentary style. In a fragmentary context you need to follow the usual conventions with closures, so redefining self though instance_eval is not a good choice. With self-contained DSL scripts you're in a different style of code to regular Ruby code and the redefinition thus doesn't cause confusion and it's worth it to remove the noisy references.