Martin Fowler's Bliki

Here's a common misconception about agile methods. It centers on the way user stories are created and flow through the development activity. The misconception is that the product owner (or business analysts) creates user stories and then put them in front of developers to implement. The notion is that this is a flow from product owner to development, with the product owner responsible for determining what needs to be done and the developers how to do it.

A justification for this approach is that this separates the responsibilities along the lines of competence. The product owner knows the business, what the software is for, and thus what needs to be done. The developers know technology and know how to do things, so they can figure out how to realize the demands of the product owner.

This notion of product owners coming up with DecreedStories is a profound misunderstanding of the way agile development should work. When we were brainstorming names at Snowbird, I remember Kent suggesting "conversational". This emphasized the fact that the heart of our thinking was of an on-going conversation between customers and developers about how a development project should proceed.

In terms of coming up with stories, what this means is that they are always something to be refined through conversation - and that developers should play an active role in helping that definition.

• spotting inconsistencies and gaps between the stories
• using technical knowledge to come up with new stories that seem to fit the product owner's vision
• seeing alternative stories that would be cheaper to build given the technological landscape
• split stories to make them easier to plan or implement

This is the Negotiable principle in Bill Wake's INVEST test for stories. Any member of an agile team can create stories and suggest modifications. It may be that just a few members of a team gravitate to writing most of the stories. That's up to the team's self-organization as to how they want that to happen. But everyone should be engaged in coming up and refining stories. (This involvement is in addition to the develpers' responsibility to estimate stories.)

The product owner does have a special responsibility. In the end the product owner is the final decider on stories, particularly their prioritization. This reflects the fact that the product owner should be the best person to judge that slippery attribute of business value. But having a final decision maker should never stop others from participating, and should not lead people astray into a decreed model of stories.

Time for another update on my DSL book's progress, since I've not been writing anything else recently.

I had my first round of technical review late in 2009 and have been incorporating comments into the current drafts. Progress on this has gone well, in large part because travel is light this time of the year. I'm also integrating my book production process into that of Pearson's.

The next visible targets are a second round of technical review and the launching of a roughcut. We're hoping to get these going in the next couple of months. The roughcut will also allow people other than official reviewers the chance to throw rocks at the text.

After that the material will be gradually readied for production. We're going to use a much more incremental process than I've used before, which will be both good and interesting. My sense at the moment that we'll see a physical book on bookshelves by the final quarter of 2010. It's currently looking at around 500 pages total in a DuplexBook split 150/350

The material currently on my web-site was last updated in June. While I've done quite a lot of detailed work on the book since, the broad structure is pretty similar, so the website gives a reasonably good picture of the scope of content.

There's been a few posts over the last couple of months about TechnicalDebt that's raised the question of what kinds of design flaws should or shouldn't be classified as Technical Debt.

A good example of this is Uncle Bob's post saying a mess is not a debt. His argument is that messy code, produced by people who are ignorant of good design practices, shouldn't be a debt. Technical Debt should be reserved for cases when people have made a considered decision to adopt a design strategy that isn't sustainable in the longer term, but yields a short term benefit, such as making a release. The point is that the debt yields value sooner, but needs to be paid off as soon as possible.

To my mind, the question of whether a design flaw is or isn't debt is the wrong question. Technical Debt is a metaphor, so the real question is whether or not the debt metaphor is helpful about thinking about how to deal with design problems, and how to communicate that thinking. A particular benefit of the debt metaphor is that it's very handy for communicating to non-technical people.

I think that the debt metaphor works well in both cases - the difference is in nature of the debt. A mess is a reckless debt which results in crippling interest payments or a long period of paying down the principal. We have a few projects where we've taken over a code base with a high debt and found the metaphor very useful in discussing with client management how to deal with it.

The debt metaphor reminds us about the choices we can make with design flaws. The prudent debt to reach a release may not be worth paying down if the interest payments are sufficiently small - such as if it were in a rarely touched part of the code-base.

So the useful distinction isn't between debt or non-debt, but between prudent and reckless debt.

There's another interesting distinction in the example I just outlined. Not just is there a difference between prudent and reckless debt, there's also a difference between deliberate and inadvertent debt. The prudent debt example is deliberate because the team knows they are taking on a debt, and thus puts some thought as to whether the payoff for an earlier release is greater than the costs of paying it off. A team ignorant of design practices is taking on its reckless debt without even realizing how much hock it's getting into.

Reckless debt may not be inadvertent. A team may know about good design practices, even be capable of practicing them, but decide to go "quick and dirty" because they think they can't afford the time required to write clean code. I agree with Uncle Bob that this is usually a reckless debt, because people underestimate where the DesignPayoffLine is. The whole point of good design and clean code is to make you go faster - if it didn't people like Uncle Bob, Kent Beck, and Ward Cunningham wouldn't be spending time talking about it.

Diving debt into reckless/prudent and deliberate/inadvertent implies a quadrant, and I've only discussed three cells. So is there such a thing as prudent-inadvertent debt? Although such a thing sounds odd, I believe that it is - and it's not just common but inevitable for teams that are excellent designers.

I was chatting with a colleague recently about a project he'd just rolled off from. The project that delivered valuable software, the client was happy, and the code was clean. But he wasn't happy with the code. He felt the team had done a good job, but now they realize what the design ought to have been.

I hear this all the time from the best developers. The point is that while you're programming, you are learning. It's often the case that it can take a year of programming on a project before you understand what the best design approach should have been. Perhaps one should plan projects to spend a year building a system that you throw away and rebuild, as Fred Brooks suggested, but that's a tricky plan to sell. Instead what you find is that the moment you realize what the design should have been, you also realize that you have an inadvertent debt. This is the kind of debt that Ward talked about in his video.

The decision of paying the interest versus paying down the principal still applies, so the metaphor is still helpful for this case. However a problem with using the debt metaphor for this is that I can't conceive of a parallel with taking on a prudent-inadvertent financial debt. As a result I would think it would be difficult to explain to managers why this debt appeared. My view is this kind of debt is inevitable and thus should be expected. Even the best teams will have debt to deal with as a project goes on - even more reason not to recklessly overload it with crummy code.

With the rise of Distributed Version Control Systems (DVCS) such as git and Mercurial, I've seen more conversations about strategies for branching and merging and how they fit in with Continuous Integration (CI). There's a bit of confusion here, particularly on the practice of feature branching and how it fits in with CI.

Simple (isolated) Feature Branch

The basic idea of a feature branch is that when you start work on a feature (or story if you prefer that term) you take a branch of the repository to work on that feature. In a DVCS, you'll do this in your personal repository, but the same kind of thing works in a centralized VCS too.

I'm going to illustrate this with a series of diagrams. I have a shared project mainline, colored blue, and two developers, colored purple and green (since the developers names are Reverend Green and Professor Plum).

I'm using labeled colored boxes (eg P1 and P2) to represent local commits on the branch. Arrows between branches represent merges between branches, the boxes are colored orange to make them stand out. In this case there are updates, say a couple of bug-fixes, applied to the mainline (presumably by Mrs Peacock). When these happen our developers merge them into their work. To give this a sense of time, I'll assume we're looking at a few days work here, with each developer committing to their local branch roughly once a day.

In order to ensure things are working properly, they can run builds and tests on their branch. Indeed for this article I'll assume that each commit and merge comes with an automated build and test on the branch it's on.

The advantage of feature branching is that each developer can work on their own feature and be isolated from changes going on elsewhere. They can pull in changes from the mainline at their own pace, ensuring they don't break the flow of their feature. Furthermore it allows the team to choose its features for release. If Reverend Green takes too long, we can release with just Professor Plum's changes. Or we may want to delay Professor Plum's feature, perhaps because we are uncertain that the feature works the way we want to release it. In this case we just tell the professor to not merge his changes into mainline until we are ready for the feature. This is called cherry-picking, the team decides which features to merge in before release.

Attractive though that picture looks, there can be trouble ahead.

Although our developers can develop their features in isolation, at some point their work does have to be integrated. In this case Professor Plum easily updates the mainline with his own changes. There's no merge here because he's already incorporated the mainline changes into his own branch (there will be a build). Things are however not so simple for Reverend Green, he needs to merge all of his changes (G1-6) with all of Professor Plum's (P1-5).

(At this point many users of DVCSs may feel I'm missing something as this is a simple, perhaps simplistic view of feature branching. I'll get to a more involved scheme later.)

I've made this a big merge box as it's a scary merge. It may be just fine, the developers may have been working on completely separate parts of the code base with no interaction, in which case the merge will go smoothly. But they may be working on bits that do interact, in which case here lye dragons.

The dragons can come in many forms, and tooling can help slay some of them. The most of obvious dragon is the complexity of merging the source code and dealing with conflicts as developers edit the same files. Modern DVCSs actually handle this rather well, indeed somewhat magically. Git has quite the reputation for dealing with complicated merges. So much so that the textual issues of merging are much better than they used to be - indeed I'll go so far as to discount textual conflicts for the purposes of this article.

The problem I worry more about is a semantic conflict. A simple example of this is that if Professor Plum changes the name of a method that Reverend Green's code calls. Refactoring tools allow you to rename a method safely, but only on your code base. So if G1-6 contain new code that calls foo, Professor Plum can't tell in his code base as he doesn't have it. You only find out on the big merge.

A function rename is a relatively obvious case of a semantic conflict. In practice they can be much more subtle. Tests are the key to discovering them, but the more code there is to merge the more likely you'll have conflicts and the harder it is to fix them. It's the risk of conflicts, particularly semantic conflicts, that make big merges scary.

This fear of big merges also acts as a deterrent to refactoring. Keeping code clean is constant effort, to do it well it requires everyone to keep an eye out for cruft and fix it wherever they see it. However this kind of refactoring on a feature branch is awkward because it makes the Big Scary Merge worse. The result we see is that teams using feature branches shy away from refactoring which leads to uglier code bases.

Continuous Integration

It's these problems that Continuous Integration was designed to solve. With Continuous Integration my diagram looks like this.

There's a lot more merging going on here, but merging is one of those things that's much easier to do frequently and small rather than rarely and large. As a result if Professor Plum is changing some code that Reverend Green relies on, the Reverend will find it early, such as when he merges in P1-2. At that point he's only got to modify G1-2 to work with the changes, rather than G1-6.

CI is effective at removing the problem of big merges, but it's also a vital communication mechanism. In this scenario the potential conflict will actually appear when Professor Plum merges G1 and realizes that Reverend Green is actively building on Plum's libraries. At this point Professor Plum can go and find Reverend Green and they can discuss how their two features interact. It may be that Professor Plum's feature requires some changes that don't mesh well with Reverend Green's changes. By looking at both their features they can come up with a better design that affects both their work-streams. With the isolated feature branches our developers don't discover this till late, probably too late to do much about it. Communication is one of the key factors in software development and one of CI's most important features is that it facilitates human communication.

It's important to note that, most of the time, feature branching like this is a different approach to CI. One of the principles of CI is that everyone commits to the mainline every day. So unless feature branches only last less than a day, running a feature branch is a different animal to CI. I've heard people say they are doing CI because they are running builds, perhaps using a CI server, on every branch with every commit. That's continuous building, and a Good Thing, but there's no integration, so it's not CI.

Promiscuous Integration

Earlier I said parenthetically that there are other ways of doing feature branching. Say Professor Plum and Reverend Green take tea together early in the cycle. While chatting they discover they are working on features that interact. At this point they may choose to integrate with each other directly, like this.

With this approach they only push to the mainline at the end, as before. But they merge frequently with each other, so this avoids the Big Scary Merge. The point here is that the primary issue with the isolated feature branching scheme is its isolation. When you isolate the feature branches, there is a risk of a nasty conflict growing without you realizing it. Then the isolation is an illusion, and will be shattered painfully sooner or later.

So is this more ad-hoc integration a form of CI or a different animal entirely? I think it is a different animal, again a key point of CI is everyone integrates to the mainline every day. Integrating across feature branches, which I shall call promiscuous integration (PI), doesn't involve or even need a mainline. I think this difference is important.

Promiscuous Integration vs Continuous Integration

So if it's different is PI better than CI, or more realistically under what circumstances is PI better than CI?

With CI, you lose the ability to use the VCS to do cherry picking. Every developer is touching mainline, so all features grow in the mainline. With CI, the mainline must always be healthy, so in theory (and often in practice) you can safely release after any commit. Having a half built feature or a feature you'd rather not release yet won't damage the other functionality of the software, but may require some masking if you don't want it to be visible in the user-interface. This can be as simple as not including a menu item in the UI to trigger the feature.

PI can provide some middle ground here. It allows Reverend Green the choice of when to incorporate Professor Plum's changes. If Professor Plum makes some core API changes in P2, then Reverend Green can import P1-2 but leave the others until Professor Plum's feature is put onto the release.

One worry with all this picking and choosing is that PI makes it really hard to keep track of who has what in their branch. In practice, it seems tooling pretty much solves this problem. DVCSs keep a clear track of changes and their origins and can figure out that when Professor Plum pulls G3 he already has G2 but doesn't have B2. I may have made mistakes drawing the diagram by hand, but tools do keep track of these things well.

On the whole, however, I don't think cherry-picking with the VCS is a good idea.

I much prefer designing the software in such a way that makes it easy to enable or disable features through configuration changes. My colleague Paul Hammant calls this Branch by Abstraction. This requires you to put some thought into what needs to be modularized and how to control that variation, but we've found the result to be far less messy that relying on the VCS.

The main thing that makes me nervous about PI is the influence on human communication. With CI the mainline acts as a communication point. Even if Professor Plum and Reverend Green never talk, they will discover the nascent conflict - within a day of it forming. With PI they have to notice they are working on interacting code. An up-to-date mainline also makes it easy for someone to be sure they are integrating with everyone, they don't have to poke around to find out who is doing what - so less chance of some changes being hidden until a late integration.

PI arose out of open-source work, and it could be that the less intensive tempo of open-source could be a factor here. In a full time job, you work several hours a day on a project. This makes it easier for features to be worked in priority. With an open source project people often put in a hour here, and the next hour a few days later. A feature may take one developer quite a while to complete while other developers with more time are able to get features into a releasable state earlier. In this situation cherry picking can be more important.

It's important to realize that the tools you use are largely independent of the integration strategy you use. Although many people associate DVCSs with feature branching, they can be used with CI. All you need to do is mark one branch on one repository as the mainline. If everyone pulls and pushes to that every day, then you have a CI mainline. Indeed with a disciplined team, I would usually prefer to use a DVCS on a CI project than a centralized one. With a less disciplined team I would worry that a DVCS would nudge people towards long lived branches, while a centralized VCS and a reluctance to branch nudges them towards frequent mainline commits. Paul Hammant may be right: "I wonder though, if a team should not be adept with trunk-based development before they move to distributed."

Summer is coming to a close, and there's a bunch of conferences coming up in the next few months..

Before the summer does end, I'll be going to Agile 2009 in Chicago. I'm not speaking at the conference, but I will be hanging around the corridors and helping host some ThoughtWorks activities. We've long had an office in Chicago and we have a lot of speakers at the conference.

In October I'll be in Europe and as usual at this time of year I'll be at JAOO. Again Rebecca Parsons and Neal Ford will be joining me for our all-day DSL tutorial. We've worked a bit on rejigging it as my book gets more stable and Rebecca and I learn from Neal's presentation skills. I'll also be giving a talk on the role of patterns in software development skills.

Rebecca and Neal will be joining me for our DSL tutorial at JAOO

I have to fly out of Aarhus pretty quickly because the next week I'll be doing a keynote for Microsoft's Patterns and Practices Summit. I haven't exactly decided what my talk will cover yet, and indeed it may be ExtemporarySpeaking.

In November I return to the Left Coast to visit Vancouver, one of my favorite cities, to speak at Agile Vancouver.

My final booking so far is to return to QCon San Francisco, where I'll be giving my talk on Three Years of Ruby, that I gave at QCons in London, Beijing, and Tokyo earlier this year.

Like many geeks I'm into photography. We geeks like photography because it provides the veneer of an artistic endeavor while allowing us to indulge in lots of technical details and spend money on expensive toys. A friend recently asked about my camera buying decisions, a question that prompted me to write them down. I got my first digital SLR a year ago. Before that I had owned a film SLR for many years, but started using digital cameras around 2000. I found the convenience of digital to be compelling and stopped using the film camera. I toyed with getting a digital SLR in 2004, but instead decided on a high end fixed lens camera - the Minolta A1. I enjoyed using it, but it conked out late in 2007. I considered a similar kind of camera, something like a Canon S5, but decided to bite the SLR bullet.

My first decision, and a critical one, was which system to buy. This is the critical decision as it's difficult (ie expensive) to reverse. Once you pick your system, you'll then commit money to it by buying lenses and the cost of switching is more than a dabbler like me can go with. I felt that the best choice was to go with the big two - Canon or Nikon. The choice between them was pretty much arbitrary, I ended up choosing Canon because a friend we occasionally vacation with has a Canon. A trifling distinction, but really the choice between the two wasn't a big one.

I'm still reasonably happy with it. One misgiving is that the technological advantage seems to have tipped in Nikon's favor over the last year, at least according to the blogs I read, but it's a tight race and Canon could well come back. I've also been recently intrigued by the new Micro Four Thirds format. Early days (and not around last year) but the small size and weight are very important to me.

With Canon as the choice, the next step was the initial choice of body and lenses. My approach was to get pretty much the cheapest body I could (the Digital Rebel XTI) because I'd rather spend more money on lenses than on the body. The whole point of SLRs is to have good lenses, so I'd rather concentrate my limited dollars there. Cameras also get upgraded much more frequently, so I'm likely to upgrade the camera in a few years while lenses stay current for much longer.

So which lenses? I forgoed the kit lens and got the camera body-only. As my main lens I went with a mega-zoom, the Sigma 18-200. Serious photographers will, probably rightly, turn their noses up at this lens. But I'm a dabbler. Most of my photos will only be seen on my screensaver or on a web page. A few get printed for a wall of our house, but only on a regular letter size printer. So I doubt that I'd appreciate the difference of a higher quality lens. Furthermore I can shoot within its limits. Reviews suggest that if you stick to f9, the quality stays pretty good. Since I'm mostly using it outside during the day, that limitation is easy to live with. As a result I tend to set my camera to aperture priority with f9, and that covers most of my shots.

The advantages of a single mega-zoom are considerable to me. Most of my photographs are taken while I'm doing something else, often with others around. I don't go out much to just shoot. In that situation even changing lenses can be a significant deterrent to getting a shot. Furthermore size and weight are a big deal when I'm travelling. While the lens isn't exactly svelte, it's much more compact that the alternative ways of getting that kind of zoom range. A final bonus is that it's image stabilized, which allows me to use it for static interiors.

The mega-zoom stays on my camera most of the time, but it wasn't the only lens I got with my camera. I also picked up the f1.8 50mm. This is an easy lens to get, very cheap, very light, very small but produces great quality. Since it's the equivalent of a 80mm on 35mm film, it's ideal for portrait photos - particularly with the f1.8 aperture. I use it a lot for shooting people in low light conditions.

I toyed with other lenses, but I wanted to get used to those two before I plonked money on any more.

After a few months with the camera I turned my eyes to a tripod. There are varying views on the net about tripods, some feel you should only use them if you really have to, some that you should use them whenever you can. I do like having one around, particularly for crepuscular shooting. I had a cheap and crummy silk tripod, but Duncan's blog persuaded me that I should get something better. I didn't go for his preferred Gitzos (beyond my budget) but I did get a light Induro tripod, together with a Really Right Stuff head and fast release clamp.

I went for the lightest setup I could get, as I wanted something that I'd actually be prepared to carry around and my camera/lens combos aren't particularly heavy. The fast release clamp was important as I'm someone who like to move around when shooting and such a clamp makes a big difference. In hindsight I wish I'd paid the extra for an L clamp, as I do find it frustrating to futz with the head when switching orientations.

It was only a month or so more before I went for another lens. A trip out to Colorado and Utah was the trigger to think about something wider than the 18-200 would go. I considered the Tokina 11-16 and the Canon 10-22, going for the latter due to it being lighter. It's a fun lens to use, allowing a few different things than what my regular lens provides. In particular what's interesting to work with is the huge depth of field you can get with an ultra wide: at 10mm you can easily get everything from a foot to infinity.

This is probably a reasonable moment to talk about filters. There's a good bit of discussion on the net about whether putting on a UV filter is worthwhile. I decided to get one for the 18-200 as it's on my camera so much, but not to get ones for my other lenses as I use them much less and am prepared to be more careful when those lenses are on the camera. For the mega-zoom I also picked up a polarizing filter, which I carry around with me all the time, but frequently forget to use.

The other issue that obsesses camera people is how to carry all this stuff. All things being equal, I like weight on my waist. So I went for a waist belt (from Tamrac, due to the double belt layout) and a Think Tank holster. I like the Think Tank's ability to extend when I have the hood on my lens. The only problem is that there are plenty of occasions when a waist belt isn't an option. The holster comes with a shoulder strap, which is fine, but I usually want the 10-22 as well. Cindy came to the rescue, sewing some straps onto the side of the holster so I can attach a lens pouch.

To keep track of my photos, and to do some post-processing, I got a copy of Apple's Aperture. (It seemed a toss up between Aperture and Lightroom.) I find it works well, better than sticking with iPhoto.

The latest lens I added to my collection is the Canon f2 100mm. I got this for shooting indoors, particularly at conferences for shooting someone on stage. In those situations I need more reach than the 50mm, but I still want a really fast aperture at a price and weight that's rather less than the serious zooms. So far I've only used the 100mm a couple of times, but have been very happy with it.

That burst of buying isn't something I expect to maintain. The quartet of lenses I have is pretty suited to my needs. There are some more I'm eyeing. The Canon 100-400mm zoom would be great for wildlife shots, but frankly we're rarely in the situation where I'd use it, so it's hard to justify its high cost. A different situation that regularly tickles my mind is cases where I'm primarily at a conference (so have the 50 and 100mm) but don't want to lug the 18-200 and want to have something wider. I could take the 10-22, but that leaves a gap and is less light than I'd like. Ironically this suggests the (now updated) kit zoom which is cheap and light. The primes less than 50mm are either too heavy, too expensive, or seem to have less quality than the kit zoom.

If you're curious, here are the results, (because there just aren't enough holiday snaps on the web.)

One of the classic cases for using a TestDouble is when you call a remote service. Remote services are usually slow and often unreliable, so using a double is a good way to make your tests faster and more stable.

When you're querying a remote service, you need to find a way to load the expected data into your double. One way to do this is to use what I'm dubbing a self-initializing fake. The basic plan is simple. The first time you call the fake it passes the call onto the actual remote service, and as it returns the data it takes and saves a copy. Further calls just return the copy. In a sense this is like a cache, but with the important difference that there is no attempt to handle cache invalidation, which is handy as that's one of the TwoHardThings.

I've called this a fake, as that seems the closest fit from the various varieties of test doubles. The other reasonable alternative is a stub, but the distinction here is that a stub needs setting up when you build the fixture, while fakes are autonomous.

The interesting thing about a self-initializing fake is how you deal with situations where the remote service changes it's response.

One time I saw this approach was with a database controlled by another application. In this case the data did change, frequently. This is unhelpful for tests, because automated tests rely on getting the same answers to the same questions. But usually tests don't care whether the data is up to date or not, so saving an old value worked just fine.

I ran into this again recently while chatting with my colleague Josh Price. In his case the remote data was supposedly static, but occasionally there were changes, which would imply that the system he was developing needed to change - usually to handle formatting issues. In this case he had a special test suite that would get all self-initializing fakes to call the remote service and check that they returned the same value that was saved.

In this case early stages of their build pipeline ran against the fake, and the last (slowest) stage ran against the service itself. One interesting problem was that the remote service required some unimportant parameters which changed from call to call but didn't alter the results. These were stripped out of the URL when the fake looked the values up from the store.

(Thanks to Josh Price, Darren Cotterill, and Gerard Meszaros for their help with this piece.)