When I start to write about a topic, I need to swiftly explain what it is I'm writing about, in this to explain what is a Domain Specific Language (DSL). I like to do this by showing a concrete example and then follow it up with a more abstract definition. So in this case I'm going to start with a concrete example to demonstrate the different forms a DSL can take. In the next chapter I'll try to generalize the definition into something more widely applicable.
Gothic Security
I have vague but persistant childhood memories of watching cheesy adventure films on TV. Often these films would be set in some old castle and feature secret compartments or passages. In order to find them our heroes would need to pull the candle holder on the top of stairs and tap the wall twice.
Let's imagine a company that decides to build security systems based on this idea. They come in an install some kind of wireless network and attach little devices that send messages when interesting things happen. So I attach a sensor to a drawer that sends the message "D2OP" when the drawer is opened. We also have little control devices that respond to command messages - so a device can unlock a door when it hears the message "D1UL".
At the centre of all this is some controller software that listens to event messages, figures out what to do, and sends command messages. The company bought a job lot of java-enabled toasters during the dotcom crash and is using them as the controllers. So whenever a customer buys a gothic security system they come in, fit the building with lots of devices and toaster with a control program written in java.
For this example, I'll focus on this control program. Each customer has their own individual needs, but once you look at a good mix of them, you soon see common patterns. Miss Grant closes her bedroom door, opens a draw, and turns on a light to access a secret compartment. Miss Shaw turns on a tap, then opens either of her two compartments by turning on correct light. Miss Smith has a secret compartment inside a locked closet inside her office. She has to close a door, take a picture off the wall, turn her desklight on three times, open the top draw of her filing cabinet - and then the closet's unlocked. If she forgets to turn the desklight off before she opens the inner compartment she wants an alarm to sound.
Although this example is deliberately whimsical, the underlying point isn't that unusual. What we have is a family of systems which share most components and behaviors, but have some important differences. In this case the way the controller sends and receives messages is the same across all the customers, but the sequence of events and commands differs. We want to arrange things so that the company can install a new system with the minimum of effort, so it must be easy for them to program the sequence of actions into the controller.
Looking at all these cases, it emerges that a good way to think about the controller is as a state machine. Each sensor sends an event that can change the state of the controller. As the controller enters a state it can send a command message out to the network.
At this point I should confess that in my writing it was really the other way around. A state machine makes a good example for a DSL, so I picked that first. I chose a gothic castle because I get bored of other state machine examples.
Miss Grant's Controller
Although my mythical company has thousands of satisfied customers, we'll focus on just one as an example - that of Miss Grant, since she's my favorite. She has a secret compartment in her bedroom that is normally locked and concealed. To open it she has to close the door, open the second draw in her chest, turn her bedside light on - and then the secret panel is unlocked for her to open.
I can represent this sequence as a state diagram.
If you haven't come across state machines yet, they are a common way of describing behavior - not useful everywhere but well suited to situations like this. The basic idea is that you think of the controller as being in different states. When you're in a particular state certain events will transition you to another state. That state will have different transitions on it, thus a sequence of events leads you from state to state. In this model actions (sending of commands) occur when you enter a state. (Other kinds of state machines place actions in different places.)
This controller is mostly a conventional and simple state machine, but there is a twist. Most of the customers' controllers have a distinct idle state that the system spends most of its time in. Certain events can jump the system back into this idle state even if they are in the middle of the more interesting parts of the state transitions, effectively resetting the model. In Miss Grant's case opening the door is such a reset event.
Introducing reset events means that the state machine I have hear isn't doesn't quite fit one of the classical state machine models. There are several variations of state machines that are pretty well known, this model starts with one of these and then the reset events add a twist that is unique to this context.
In particular you should note that reset events aren't
strictly necessary to express the Miss Grant's controller. As an
alternative I could just add a transition to every state,
triggerred by doorOpened leading to the idle
state. The notion of a reset event is useful because it
simplifies the diagram.
The State Machine Model
Once the team has decided that a state machine is a good abstraction to use to specify the how the controllers work, the next step is to ensure that abstraction is put into the software itself. If people want to think about controller behavior with events, states, and transitions - then we want that vocubulary to be present in the software code too. This is essentially the Domain Driven Design principle of Ubiquitous Languge - that is we construct a shared language between the domain people (who describe how the building security should work) and programmers.
I hope most readers will at this point be thinking about creating a bunch of java classes to capture controller behavior. Essentially we want to create a Domain Model for the state machine.
[TBD: Add reset event association to class diagram]
The controller communicates with the devices by receiving event messages and sending command messages. These are both four letter codes that are sent through the communication channels. I want to refer to these in the controller code with symbolic names, so I create event and command classes with a code and a name. I keep them as separate classes (with a superclass) as they play different roles in the controller code.
class AbstractEvent...
private String name, code;
public AbstractEvent(String name, String code) {
this.name = name;
this.code = code;
}
public String getCode() { return code;}
public String getName() { return name;}
public class Command extends AbstractEvent
public class Event extends AbstractEvent
The state class keeps track of the commands that it will send and its outbound transitions.
class State...
private String name;
private List<Command> actions = new ArrayList<Command>();
private Map<String, Transition> transitions = new HashMap<String, Transition>();
public void addTransition(Event event, State targetState) {
transitions.put(event.getCode(), new Transition(this, event, targetState));
}
class Transition...
private final State source, target;
private final Event trigger;
public Transition(State source, Event trigger, State target) {
this.source = source;
this.target = target;
this.trigger = trigger;
}
public State getSource() {return source;}
public State getTarget() {return target;}
public Event getTrigger() {return trigger;}
public String getEventCode() {return trigger.getCode();}
The state machine holds on to its start state.
class StateMachine...
private State start;
public StateMachine(State start) {
this.start = start;
}
Any other states in the machine are then those that are reachable from this state.
class StateMachine...
public Collection<State> getStates() {
List<State> result = new ArrayList<State>();
gatherForwards(result, start);
return result;
}
private void gatherForwards(Collection<State> result, State start) {
if (start == null) return;
if (result.contains(start)) return;
else {
result.add(start);
for (State next : start.getAllTargets()) {
gatherForwards(result, next);
}
return;
}
}
class State...
Collection<State> getAllTargets() {
List<State> result = new ArrayList<State>();
for (Transition t : transitions.values()) result.add(t.getTarget());
return result;
}
To handle reset events I keep a list of them on the state machine.
class StateMachine...
private List<Event> resetEvents = new ArrayList<Event>();
public void addResetEvents(Event... events) {
for (Event e : events) resetEvents.add(e);
}
I don't need to have a separate structure for reset events like this. I could handle this by simply declaring extra transitions on the state machine like this
class StateMachine...
private void addResetEvent_byAddingTransitions(Event e) {
for (State s : getStates())
if (!s.hasTransition(e.getCode())) s.addTransition(e, start);
}
I prefer explicit reset events on the machine becuase that better expresses the intention of what I'm trying to do. While it does complicate the machine a bit, it keeps the clarity of my intention of how a general machine is supposed to work, as well as keeping the intention of how a particular machine is defined.
With the structure out of the way, now lets move on to the behavior. As it turns out, it's really quite simple. The controller has a handle method that takes the event code it receives from the device.
class Controller...
private State currentState;
private StateMachine machine;
public CommandChannel getCommandChannel() {
return commandsChannel;
}
protected CommandChannel commandsChannel;
public void handle(String eventCode) {
if (currentState.hasTransition(eventCode))
transitionTo(currentState.targetState(eventCode));
else if (machine.isResetEvent(eventCode))
transitionTo(machine.getStart());
// ignore unknown events
}
private void transitionTo(State target) {
currentState = target;
currentState.executeActions(commandsChannel);
}
class State...
public boolean hasTransition(String eventCode) {
return transitions.containsKey(eventCode);
}
public State targetState(String eventCode) {
return transitions.get(eventCode).getTarget();
}
public void executeActions(CommandChannel commandsChannel) {
for (Command c : actions) commandsChannel.send(c.getCode());
}
class StateMachine...
public boolean isResetEvent(String eventCode) {
return resetEventCodes().contains(eventCode);
}
private List<String> resetEventCodes() {
List<String> result = new ArrayList<String>();
for (Event e : resetEvents) result.add(e.getCode());
return result;
}
It ignores any events that are not registered on the state. For any events that are recognized, it transitions to the target state and executes any commands defined on that target state.
Programming Miss Grant's Controller
Now I've implemented the state machine model, I can now program Miss Grant's controller like this.
Event doorClosed = new Event("doorClosed", "D1CL");
Event drawOpened = new Event("drawOpened", "D2OP");
Event lightOn = new Event("lightOn", "L1ON");
Event doorOpened = new Event("doorOpened", "D1OP");
Event panelClosed = new Event("panelClosed", "PNCL");
Command unlockPanelCmd = new Command("unlockPanel", "PNUL");
Command lockPanelCmd = new Command("lockPanel", "PNLK");
Command lockDoorCmd = new Command("lockDoor", "D1LK");
Command unlockDoorCmd = new Command("unlockDoor", "D1UL");
State idle = new State("idle");
State activeState = new State("active");
State waitingForLightState = new State("waitingForLight");
State waitingForDrawState = new State("waitingForDraw");
State unlockedPanelState = new State("unlockedPanel");
StateMachine machine = new StateMachine(idle);
idle.addTransition(doorClosed, activeState);
idle.addAction(unlockDoorCmd);
idle.addAction(lockPanelCmd);
activeState.addTransition(drawOpened, waitingForLightState);
activeState.addTransition(lightOn, waitingForDrawState);
waitingForLightState.addTransition(lightOn, unlockedPanelState);
waitingForDrawState.addTransition(drawOpened, unlockedPanelState);
unlockedPanelState.addAction(unlockPanelCmd);
unlockedPanelState.addAction(lockDoorCmd);
unlockedPanelState.addTransition(panelClosed, idle);
machine.addResetEvents(doorOpened);
I look at this last bit of code as quite different in nature to the previous peices. The earlier code described how to build the state machine model, this last bit of code is about how to configure that model for one particular controller. You often see divisions like this. On the one hand is library, framework, or component implementation code; on the other is configuration or component assembly code. Essentially it is the separation of common code from variable code. We structure the common code in a set of components that we then configure for different purposes.
[TBD: Consider figure here to bring out multiple configuration code using single library]Here is another way of representing that configuration code.
<stateMachine start = "idle">
<event name="doorClosed" code="D1CL"/>
<event name="drawOpened" code="D2OP"/>
<event name="lightOn" code="L1ON"/>
<event name="doorOpened" code="D1OP"/>
<event name="panelClosed" code="PNCL"/>
<command name="unlockPanel" code="PNUL"/>
<command name="lockPanel" code="PNLK"/>
<command name="lockDoor" code="D1LK"/>
<command name="unlockDoor" code="D1UL"/>
<state name="idle">
<transition event="doorClosed" target="active"/>
<action command="unlockDoor"/>
<action command="lockPanel"/>
</state>
<state name="active">
<transition event="drawOpened" target="waitingForLight"/>
<transition event="lightOn" target="waitingForDraw"/>
</state>
<state name="waitingForLight">
<transition event="lightOn" target="unlockedPanel"/>
</state>
<state name="waitingForDraw">
<transition event="drawOpened" target="unlockedPanel"/>
</state>
<state name="unlockedPanel">
<action command="unlockPanel"/>
<action command="lockDoor"/>
<transition event="panelClosed" target="idle"/>
</state>
<resetEvent name = "doorOpened"/>
</stateMachine>
This style of representation should look familiar to most readers, I've expressed it as an XML file. There are several advantages to doing it this way. One obvious reason is that now we don't have to compile a separate java program for each controller we put into the field - instead we can just compile the state machine components plus an appropritate parser into a common jar, and ship the xml file to be read when the machine starts up. Any changes to the behavior of the controller can be done without having to distribute a new jar. (We do, of course, pay for this in that any mistakes in the syntax of the configuration can only be detected at run time.)
A second advantage is in the expressiveness of the file itself. We no longer need to worry about the details of making the various connections through variables. Instead we have a more declarative approach that in many ways reads much more clearly. We're also limited in that we can only express configuration in this file - limitations like this often are helpful because they can reduce the chances for people making mistakes in the component assembly code.
These advantages are why so many frameworks in Java and C# are configured with XML configuration files. These days it sometimes feels that you're doing more programming with XML than you are with main programming language.
Here's another version of the configuration code.
events
doorClosed D1CL
drawOpened D2OP
lightOn L1ON
doorOpened D1OP
panelClosed PNCL
end
resetEvents
doorOpened
end
commands
unlockPanel PNUL
lockPanel PNLK
lockDoor D1LK
unlockDoor D1UL
end
state idle
actions {unlockDoor lockPanel}
doorClosed => active
end
state active
drawOpened => waitingForLight
lightOn => waitingForDraw
end
state waitingForLight
lightOn => unlockedPanel
end
state waitingForDraw
drawOpened => unlockedPanel
end
state unlockedPanel
actions {unlockPanel lockDoor}
panelClosed => idle
end
This is code, although not in a syntax that's familiar to you. In fact it's a custom syntax that I made up for this example. I think it's a syntax that's easier to write, and above all easier to read, than the XML syntax. It's terser and avoids a lot of the quoting and noise characters that the XML suffers from. You probably wouldn't have done it exactly the same way, but the point is that you can construct whatever syntax you and your team prefers. You can still load it in at runtime (like the XML) but you don't have to (as you don't with the XML) if you want it at compile time.
This language is a Domain Specific Language, and shares many of the characteristics of DSLs. Firstly it's suitable only for a very narrow purpose - it can't do anything other than configure this particular kind of state machine. As a result the DSL is very simple - there's no facility for control structures or anything else. It's not even Turing complete. You couldn't write a whole application in this language - all you can do is describe one small aspect of an application. As a result the DSL has to be combined with other languages to get anything done. But the simplicity of the DSL means it's easy to edit and process.
Now look again at the XML representation. Is this a DSL? I would argue that it is. It's wrapped in an XML carrier syntax - but it's still a DSL. This example thus raises a design issue - is it better to have custom syntax for a DSL or an XML syntax? The XML syntax can be easier to parse since people are so familiar with parsing XML. (But as it happened it took me the about the same amount of time to write the parser for the custom syntax as it did for the XML.) I'd contend that the custom syntax is much easier to read, at least in this case. But however you view this choice the core trade-offs around DSLs are the same. Indeed you can argue that most XML configuration files are essentially DSLs.
Now look at this code. Does this look like a DSL for this problem?
event :doorClosed, "D1CL"
event :drawOpened, "D2OP"
event :lightOn, "L1ON"
event :doorOpened, "D1OP"
event :panelClosed, "PNCL"
command :unlockPanel, "PNUL"
command :lockPanel, "PNLK"
command :lockDoor, "D1LK"
command :unlockDoor, "D1UL"
resetEvents :doorOpened
state :idle do
actions :unlockDoor, :lockPanel
transitions :doorClosed => :active
end
state :active do
transitions :drawOpened => :waitingForLight,
:lightOn => :waitingForDraw
end
state :waitingForLight do
transitions :lightOn => :unlockedPanel
end
state :waitingForDraw do
transitions :drawOpened => :unlockedPanel
end
state :unlockedPanel do
actions :unlockPanel, :lockDoor
transitions :panelClosed => :idle
end
It's a bit noisier than the custom language earlier, but still pretty clear. Readers who have similar language likings to me will probably know that it's Ruby. Ruby gives me a lot of syntactic options that makes for more readable code, so I can make it look very similar to the custom language.
Ruby developers would consider this code to be a DSL. I use a subset of the capabilities of Ruby and capture the same ideas as our XML and custom syntax. Essentially I'm embedding the DSL into ruby, using a subset of ruby as my syntax. To an extent this is more a matter of attitude than of anything else. I'm choosing to look at the Ruby code through DSL glasses. But it's a point of view with a long tradition - Lisp programmers often think of creating DSLs inside Lisp.
This brings me to pointing out that there are two kinds of textual DSLs: which I call external and internal DSLs. An External DSL is a domain specific language represented in a separate language to the main programming language it's working with. This language may be a custom syntax, or it may follow the syntax of another representation (like XML). An Internal DSL is DSL expressed within the syntax of a general purpose language. It's a stylized use of that language for a domain specific purpose.
You may also hear the term embedded DSL as a synonym for internal DSL. Although it is fairly widely used, I avoid this term because you also hear "embedded language" applied to scripting languages embedded within applications: such as VBA in Excel or Scheme in the Gimp. So I use internal DSL to avoid confusion.
Now think again about the original java configuration code - is this a DSL? I would argue that it isn't. That code feels like stitching together with an API, while the ruby code above has more the feel of a declarative language. Does this mean you can't do an internal DSL in Java? How about this?
public class BasicStateMachine extends StateMachineBuilder {
Events doorClosed, drawOpened, lightOn, panelClosed;
Commands unlockPanel, lockPanel, lockDoor, unlockDoor;
States idle, active, waitingForLight, waitingForDraw, unlockedPanel;
ResetEvents doorOpened;
protected void defineStateMachine() {
doorClosed. code("D1CL");
drawOpened. code("D2OP");
lightOn. code("L1ON");
panelClosed.code("PNCL");
doorOpened. code("D1OP");
unlockPanel.code("PNUL");
lockPanel. code("PNLK");
lockDoor. code("D1LK");
unlockDoor. code("D1UL");
idle
.actions(unlockDoor, lockPanel)
.transition(doorClosed).to(active)
;
active
.transition(drawOpened).to(waitingForLight)
.transition(lightOn). to(waitingForDraw)
;
waitingForLight
.transition(lightOn).to(unlockedPanel)
;
waitingForDraw
.transition(drawOpened).to(unlockedPanel)
;
unlockedPanel
.actions(unlockPanel, lockDoor)
.transition(panelClosed).to(idle)
;
}
}
It's formatted oddly, and uses some unusual programming conventions, but it is valid Java. It's java written in what is these days called a Fluent Interface style. A Fluent Interface is an API that's designed to read like an internal DSL. This I would call a DSL - although it's more messy than the ruby DSL it still has that declarative flow that a DSL needs.
What makes a fluent interface different to a normal API? This is a tough question that I'll spend more time on later), but it comes down to a rather fuzzy notion of a language-like flow. Given this distinction it's useful to have a name for a non-fluent API - I'll use the term command-query API.
Languages and Model
There's an important inter-relationship here between the various DSLs and the underlying state-machine model. To implement each of these languages I wrote code that translated from expressions in the DSL into calls on the command-query interface of the model. So while I was parsing the custom syntax version and came across
commands unlockPanel PNUL[TBD: Consider figure to illustate DSL parsed into objects]
I would create a new command object (new
Command("unlockPanel", "PNUL")) and keep it to one side (in a
Symbol Table) so that when I saw "actions
{unlockPanel" I could add it to the appropriate state (using
addAction). As a result each DSL I've shown you created
the same configuration of objects in the model.
The model, as I discussed earlier, is the engine that provides the behavior of the state-machine. So once we have a populated model, we have a running program whose behavior is encoded in the inter-relationships between the objects in that model. This style is often called an Adaptive Model, because in order to understand the behavior of the state machine you can't just look at the code, you also have to look at the way object instances are wired together. Of course this is always true to some extent, any program gives different results with different data, but there is a sense of a greater difference here as the presence of the state objects alters the behavior of the system to a significantly greater degree.
When people discuss a programming language you often hear them talk about syntax and semantics. The syntax captures the legal expressions of the program, what in the custom syntax DSL is captured by the grammar. The semantics of a program is what it means, that is what it does when it executes. In this case it is the model that defines those semantics - which is why I will refer to it as a Semantic Model.
In this example the Semantic Model is an object model. A Semantic Model can also take other forms. It can be a pure data structure with all behavior in separate functions. I would still refer to it as an Adaptive Model, because the data structure defines the program's behavior.
Looking at it from this point of view, the DSL merely acts as a mechanism for expressing how the model is configured. Much of the benefits of using this approach comes from the model rather than the DSLs. The fact that I can easily configure a new state machine for a customer is a property of the model, not the DSL. The fact that I can make a change to a controller at run-time, without compiling, is a feature of the model, not the DSL. The fact I'm reusing code across multiple installations of controllers is a property of the model, not the DSL. Hence the DSL is merely a thin facade over the model.
A model provides many benefits without any DSLs present. As a result we use them all the time. We use libraries and frameworks to wisely avoid work. In our own software we construct our own models, building up abstractions that allow us to program faster. Good models, whether published as libraries and frameworks or just serving our own code, can work just fine without any DSL in sight.
But DSLs can enhance the the capabilities of a model. The right DSL makes it easier to understand what a particular state machine does. Some DSLs allow you to configure the model at run time. DSLs are thus a useful adjunct to some models.
In discussing this example I described a circumstance where the model was built first, and then I layered a DSL over the model to help manipulate it. I described it that way becuase I think that's an easy way to understand how DSLs fit into software development. Although the model-first case is a common one, it isn't the only one. In a different scenario you talk with the domain experts and posit that a state machine approach is something they understand. You then work with the domain experts to create a DSL that they can understand. In this case you build the DSL and model simultaneously.
Using Code-Generation
In my discussion so far, I process the DSL by populating the Semantic Model and then execute the Semantic Model to provide the behavior that I want from the controller. This approach is what's known in language circles as interpretation. When we interpret some text, we parse it and immediately produce the result that we want from the program. (Interpret is a tricky word in software circles, since it carries all sorts of connotations for people, however I'll use it strictly to mean this form of immediate execution.)
In the language world, the alternative to interpretation is compilation. With compilation, we parse some program text and produce an intermediate output, which is then separately processed to provide the behavior we desire. In the context of DSLs the compilation approach is usually referred to as code-generation. In this case this might mean generating some java code to represent the particular behavior of Miss Grant's controller.
[TBD: Consider figure to illustrate interpretation vs compilation]Code generation is often awkward in that it often pushes you to an extra compilation. To build your program you have to first compile the state framework and the parser, then run the parser to generate the source code for Miss Grant's controller, then compile that generated code. This makes your build process much more complicated.
However an advantage of code generation is that there's no reason why you have to generate code in same programming language that you use for the parser. In this case you can avoid the second compilation step by generating code for a dynamic language such as javascript or jruby. Code generation is also useful when you want to use DSLs with a language platform that doesn't have the tools for DSL support. I've come across recent projects that generate code for MathCAD, SQL, and COBOL.
Many writings on DSLs focus on code-generation, even to the point of making code-generation the primary aim of the exercise. As a result you can find articles and books extolling the virtues of code-generation. In my view, however, code-generation is merely an implementation mechanism, one that isn't actually needed in most cases. Certainly there are plenty of times when you must use code-generation, but there are even more plenty of times when you don't need it.
Using code-generation is one case where many people don't use a Semantic Model. In this case you parse the input text and directly produce the generated code. Although this is a common way of working with code-generated DSLs, it isn't one I reccommend for any but the very simplest cases. Using a Semantic Model allows me to separate the parsing, the execution semantics, and the code generation into separate problems. This separation makes the whole exercise a lot simpler. It also allows me to change my mind. I can change my DSL from an internal to an external DSL (say) without altering my code-generation routines. Similarly I can easily generate multiple outputs without complicating my parse. I can also use both an interpreted model and code generation off the same Semantic Model.
As a result for almost all of this book, I'm going to assume a Semantic Model is present and the centre of the DSL effort.
Using Language Workbenches
The two styles of DSL I've shown so far (internal and external) are the traditional ways of thinking about DSLs. They may not be as widely understood and used as they should be, but they have a long history and moderately wide usage. As a result the rest of this book concentrates on getting you started with these approaches using tools that are mature and easy to obtain.
But there is a whole new category of tools on the horizon that could change the game of DSLs significantly: tools I call Language Workbenches. A Language Workbench is tool designed to help people create new DSLs, together with high quality tooling required to use those DSLs effectively.
One of the big disadvantages of using an external DSL is that you're stuck with relatively limited tooling. Setting up syntax highlighting with a text editor is about as far as most people go. While you can argue that the simplicity of a DSL and the small size of the scripts means that may be enough, there's also an argument for the kind of sophisticated tooling that modern Post Intelli J IDEs support. Language Workbenches make it easy not just to define a parser, but also to define a custom editing environment for that language.
All of this is valuable, but the truly interesting aspect of language workbenches is that they allow a DSL designer to go beyond the the traditional text-based source editing, to different forms of language. The most obvious example of this is support for diagrammatic languages, which would allow me to specify the secret panel state machine directly with a state transition diagram.
Figure 3: The secret panel state machine displayed in the MetaEdit language workbench. (source MetaCase)
Not just does a tool like this allow you to have diagrammtic languages, it also allows you to look at a DSL script from multiple perspectives. In Figure 3 there is a diagram, but also lists of states and events, and a table to enter the event codes (which could be ommitted from the diagram if there's too much clutter there).
This kind of multi-pane visual editing environment has been around for a while in lots of tools, but it's been a lot of effort to build something like this for yourself. One promise of language workbenches is that they make it quite easy to do this, certainly I was easily able to put together a similar example to Figure 3 quite quickly on my first play with the MetaEdit tool. The tools allows me to define the Semantic Model for state machines, define the graphical and tabular editors in Figure 3 and write a code generator from the Semantic Model.
However, while such tools certainly look good, many developers are naturally suspicious of such doodleware tools. There are some very pragmatic reasons why a textual source representation makes sense. As a result other tools head in that direction, providing post-IntelliJ style capabilities such as syntax-directed editing, symbol completion and the like to textual languages.
My own suspicion here is that if language workbenches really take off, the languages they'll produce aren't anything like what we consider to be a programming language. One of the common benefits of tools like this is that they allow non-programmers to program. I often sniff at that notion by pointing out that this was the original intent of COBOL. Yet I must also acknowledge a programming environment that has been extremely successful in providing programming tools to non-programmers who program without thinking of themselves of programming - spreadsheets.
In programming language terms spreadsheets are based on a quite unusual computational model. Their appeal comes from a very deep integration of the notions of language and tool. Thus it's no surprise that Charles Simonyi combines both a history of development of these kinds of user tools with a long history of developing ideas in language workbenches. As a result I think that language workbenches have a remarkable potential. If they fulfill this they could entirely change the face of software development.
[TBD: Tie into ideas on Illustrative Programming]This potential, however profound, is still somewhat in the future. It's still early days for language workbenches with new approaches appearing regularly and older tools still subject to deep evolution. As a result I don't have that much to say about them here, as I think they will change quite dramatically during the hoped-for lifetime of this book. But I do have a chapter on them at the end, as I think they are well worth keeping an eye on.
Visualization
One the great advantages of using a Language Workbench is that this enables you to a wider range of representations of the DSL, in particular graphical representations. However even with a textual DSL you can obtain a diagrammatic representation. Indeed we saw this very early on in this chapter. When looking at Figure 1 it might have struck you that the diagram was not as neatly drawn as I usually do. The reason for this is that I didn't draw the diagram, I generated it automatically from the Semantic Model of Miss Grant's controller. Not just do my state machine classes execute, they also are able to render themselves using the dot language.
The dot langauge is part of the GraphViz package, which is an open-source tool that allows you to describe mathematical graph structures (nodes and edges) and then automatically plot them. It figures out how to lay out the graph, you just tell it what the nodes and edges are, what shapes to use, and some other hints.
Using a tool like GraphViz is extremely helpful for many kinds of DSLs because it gives another representation. This visualization representation is similar to the DSL itself in that it allows a human to understand the model. The diference between a visualization and the source is that it isn't editable - however it can provide options that are too hard in an editable form, such as a diagram like this.
[TBD: Consider block diagram to show how visualization fits into model and code gen ideas]Visualizations don't have to be graphical. I often use a simple textual visualization to help me debug while I'm writing a parser. I've seen people generate visualizations in Excel to help communicate with domain experts. The point is that once you have done the hard work of creating a Semantic Model adding visualizations is really easy. You'll note here that the visualizations are produced from the model, not the DSL, so you can do this even if you aren't using a DSL to populate the model.