One of the tricky things about writing about external DomainSpecificLanguages is that I'm walking through territory already heavily tracked by the programming languages community. Programming language research has always been a popular area of academic activity, and I'm the first to admit that I don't have anywhere near the depth in this topic as many people who've been studying in this space for years. So inevitably the question comes up as to why such a noob as me thinks he can write a book in this well trodden ground?

A while ago I was chatting with a post-doc on his way to an academic career. He was asking me about research topics wanting my input as he felt I could inform him on what would be research of practical use. I wasn't very helpful, but I did mention that the best way to do this would be to spend some time in industry to get a feel of how software development works in the wild and what problems could do with some research effort. His answer to this thought was very troubling.

Will DSLs allow business people to write software rules without involving programmers?

When people talk about DSLs it's common to raise the question of business people writing code for themselves. I like to apply the COBOL inference to this line of thought. That is that one of the original aims of COBOL was to allow people to write software without programmers, and we know how that worked out. So when any scheme is hatched to write code without programmers, I have to ask what's special this time that would make it succeed where COBOL (and so many other things) have failed.

TechnicalDebt is a very useful concept, but it raises the question of how do you measure it? Sadly technical debt isn't like financial debt, so it's not easy to tell how far you are in hock (although we seem to have had some trouble with measuring the financial kind recently).

One of the features of the new world of services that SOA-gushers promoted was the notion of registries. Often this was described in terms of automated systems that would allow systems to automatically look up useful services in a registry and bind and consume those services all by themselves.

Well computers may look clever occasionally, but I didn't particularly buy that idea. While there might the be odd edge case for automated service lookup, I reckon twenty-two times out of twenty it'll be a human programmer who is doing the looking up.

A few years ago I heard programming language people talk about the “Nuclear Winter” in languages caused by Java. The feeling was that everyone had so converged on Java's computational model (C# at that point seen as little more than a rip-off) that creativity in programming languages had disappeared. That feeling is now abating, but perhaps a more important thaw that might be beginning - the longer and deeper freeze in thinking about databases.

At QCon San Francisco 2008 Rebecca Parsons and I gave a talk about how agile approaches work with enterprise architecture groups. At the moment there's a lot of distrust and conflict between agile project teams and architecture groups. We dig into why this is so, and explore ways that these groups can work together.

Let's imagine a pretty world of SOA-happiness where the computing needs of an enterprise are split into many small applications that provide services to each other to allow effective collaboration. One fine morning a consumer service needs some information from a supplier service. The twist is that although the supplier service has the necessary data and processing logic to get this information, it doesn't yet expose that information through a service interface. The supplier has a potential service, but it isn't actually there yet.

A few years ago I was talking with a client who told me something he didn't like about the agile approach we were using: “it doesn't feel right to have these difficulties this early in the project”. Contrary to his reaction, in my mind this early pain is one of the great benefits of an agile or indeed any iterative development process.

Oslo is a project at Microsoft, of which various things have been heard but with little details until this week's PDC conference. What we have known is that it has something to do with ModelDrivenSoftwareDevelopment and DomainSpecificLanguages.

A Microsoft Channel 9 interview of me, my colleague Neal Ford, and Jeffery Snover (creator of PowerShell). The general topic is that of DSLs - Neal and I had just finished a tutorial on the topic at JAOO 2008 and had some good conversations with Jeffery.

Thoughtworks often organizes “Quarterly Technology Briefings” - open talks for the community in cities where we have offices. In this QTB in Toronto, Scott Shaw and I talk about how to build new relationships between IT and business. It explains why we think IT departments should be disbanded.

Requirements are the things that you should discover before starting to build your product. Discovering the requirements during construction, or worse, when you client starts using your product, is so expensive and so inefficient, that we will assume that no right-thinking person would do it, and will not mention it again.

-- Suzanne and James Robertson

Agile methods violate this underlying assumption by intending to discover the 'requirements' during construction and after delivery. But even this cavalier disregard of the above sage advice is nothing compared to what many leading web sites do these days. These sites explore requirements by observing what the users do on their sites and using that information to generate ideas for new features along the following lines:

Can SOA be done with an agile approach?

I was asked to put together a discussion of DSLs for non-technical types. Maybe I've been reading too much Stephen O'Grady, but I felt an irresistible urge to do it in a Q and A manner. So here it comes.

Language Workbench is a term I coined in 2005 to describe a new class of software development tool, designed to build software through a rich environment of multiple, integrated, DomainSpecificLanguages. These tools are still quite a way away from being mainstream, but development on them continues and continues to be interesting. They are one of the few things I feel could significantly change the programming landscape.

What is the connection between ModelDrivenSoftwareDevelopment (MDSD) and DomainSpecificLanguages (DSLs)?

It's pretty common to see the term “DSL” crop up in the context of MDSD. Indeed some MDSD people seem to think that DSLs only exist within the MDSD world. I've been writing a lot on DSLs recently for my book, but so far I haven't really touched on the MDSD angle much Instead I've concentrated on DSLs role in more conventional programming. DSLs exist in both the textual language and MDSD worlds and play pretty much the same role for both.

Model Driven Software Development (MDSD) is a style of software development that considers itself as an alternative to the traditional style of programming. The approach centers itself on building models of a software system. These models are typically made manifest through diagrammatic design notations - the UML is one option. The idea is that you use these diagrams, to specify your system to a modeling tool and then you generate code in a conventional programming language.

Like any profession, software development has it's share of oft-forgotten activities that are usually ignored but have a habit of biting back at just the wrong moment. One of these is data migration.

I'm thinking of using agile software development - but should I use Lean software development instead?

One of the biggest issues with media websites is dealing with high amounts of traffic. Media is all about getting eyeballs, but if you get too many hits at once, slow performance can cause problems and damage your reputation. This problem is exacerbated by the bursty nature of this web traffic. You can be cruising along at a manageable rate, then get hit with a big news story which causes a big spike. One of our clients have seen spikes of two orders of magnitude in a matter of a couple of minutes.

A common phrase that's bandied about when talking about DomainSpecificLanguages (or indeed any computer language) is that of noisy syntax. People may say that Ruby is less noisy than Java, or that external DSLs are less noisy than internal DSLs. By Syntactic Noise, what people mean is extraneous characters that aren't part of what we really need to say, but are there to satisfy the language definition. Noise characters are bad because they obscure the meaning of our program, forcing us to puzzle out what it's doing.

I talk quite a bit with people about DomainSpecificLanguages these days and a common reaction I get to external DSLs is that it's hard to write a parser. Indeed one of the justifications for using XML as the carrier syntax for an external DSL is that “you get the parser for free”. This doesn't jive with my experience - I think parsers are much easier to write than most people think, and they aren't really any harder than parsing XML.

The basic idea of a Domain-Specific Language (DSL) is a computer language that's targeted to a particular kind of problem, rather than a general purpose language that's aimed at any kind of software problem. Domain-specific languages have been talked about, and used for almost as long as computing has been done.

For nth, and I'm sure not last time, I'm sliding into a conversation about defining practices, labeling some of them as “best”, and probably the C-word (certification). It's a familiar discussion, and although we've barely started it, I can predict much of where it will go. It's driven by a perfectly reasonable desire to identify who are the better software developers, and how existing developers can improve their abilities.

My colleague Jim Webber has built quite a reputation for taking a lightweight and business-oriented approach to integration in the enterprise. He also has a reputation for being a very robust and entertaining speaker. So I was as nervous as I was excited to share the stage with him for a keynote at QCon 2008. He put together a wonderfully funny presentation with some serious tidbits of meat woven through it. We then just dove in and did it - possibly helped by the pre-talk pint. We talk about the history of Enterprise Integration, the growth of systems that think they are strong but are really just fat, the role of agile thinking, the influence of the web (including Jim's unique theory on why it was invented), and how this leads to Guerilla SOA.

One of the commonly accepted beliefs in the software world is that talented programmers are more productive. Since we CannotMeasureProductivity this is a belief that cannot be proven, but it seems reasonable. After all just about every human endeavor shows some people better than others, often markedly so. It's also commonly observed by programmers themselves, although it always seems to be remarked on by those who consider themselves to be in the better talented category.

Source based programing environments hold the definition of a system in a set of source files which act as both an editable and storage representation of the system definition. These sources are then transformed to an executable representation that actually runs. Source based code is the most familiar form around today in contrast to in contrast to ProjectionalEditing (where I discuss the two in more detail).

Imagine a hiring situation. There's two candidates both with a few years of experience. In the blue corner we have someone with good broad design skills in the style of design that you favor (in my case that would be things like DRY, judicious use of patterns, TDD, communicative code etc, but the actual list isn't important - just that it's what you favor). However she knows nothing of the particular platform technology that you're using. In the red corner we have someone who has little knowledge (or interest) in those issues, but knows your platform really well - edge cases in the language, what libraries are available, fingers move naturally over the tools. Assume all else about them is equal (which it never is except for thought experiments like this) and that your team doesn't have any gaping holes that this candidate might fill. Which one would you prefer?

An alternative to SourceEditing is the idea that the core definition of a system should be held in a model and edited through projections.