Martin Fowler

InfoQ Interview with Paulo Caroli and me at Agile Brazil

2010-08-25T15:30:00-04:00

In June I gave a talk at Agile Brazil. During the conference my colleague Paulo Caroli and I were grabbed to give an interview for InfoQ Brazil. We touched on several topics: acceptance testing, polyglot programming, my DSL book, REST, and continuous delivery.

DSL copy-edited text in roughcut

2010-08-17T17:56:00-04:00

Over the last couple of months my DSL book has been through production and will shortly head off to the printers. Part of this process is copy-editing, and I’ve now updated the rough-cut to use the copy-edited text. So now any typos you see in the rough-cut are genuine errors, thus I won’t ignore them any more.

If you’re interested in hearing more about DSLs, Neal Ford and I will be giving our full-day tutorial on DSLs at JAOO again this year. We’re hoping the book will make its conference debut there as well.

Bliki: Agile2010

2010-08-16T13:49:00-04:00

Last week I attended the Agile 2010 conference in Orlando. Agile 20xx is the major US agile-oriented conference whose roots go back to XP Universe and the Agile Development Conference. I've not been a regular attender of the main agile conferences, but I did go last year as well. Rather than make an attempt at a consolidated description, here are a few scattered impressions.

Elizabeth Hendrickson (aka testobsessed)

Attendance was 1400, more than 2009, but around the same as 2008. So although it has been hit by the Great Recession it's weathered it reasonably well.
I was delighted to see Elizabeth Hendrickson and Liz Keogh win the Gordon Pask award.
My sense was that most people were relatively new to agile, which is in line with my sense in 2009.
There was a much higher proportion of women than I usually see at software conferences.
There were lots of talks going on, which felt overwhelming to me: 16 stages (tracks) with 214 talks.
I was pleased to see the tutorial on Continuous Delivery that Jez Humble gave with me seemed well-received. (We currently have it booked again for QCon and OOP.)
Most of the emphasis was on team collaboration and soft skills. This led to some criticism from people on the programming side.
My unscientific sample of talks I attended showed a higher proportion of badly presented talks than I would like.
It seems there's a movement to re-create XP Universe for next year.
I prefer a conference in a real city to one in a holiday resort.
This year marks ten years from the first agile-oriented conference.
The conference went smoothly - which is a testament to the hard work of all the organizers. This is particularly so since they had to deal with moving the conference after the flooding in Nashville. Thank You.

Liz Keogh (aka lunivore)

I have mixed feelings about the complaints on the lack of programming topics. Certainly programming plays a central role in software development and attempts to marginalize it correlate well with dead-ends. But programming is part of a very interconnected system which involves lots of pieces - and making these interconnections work requires exactly these kinds of "soft and woolly" skills. There is a real tendency for programmers to look inward and obsess on technical issues rather than engaging with those outside programming. One of the things I like about XP is that it recognizes this: blending technical excellence with collaboration.

I would hate to see programming detached from the agilexx conference with programmers going to different events. There's a need for places where the intermingling can occur. The JAOO and QCon conferences seem to do a very good job of this - as well as having a better quality control over the talks themselves. Their format isn't universal, and there's room for more options. Our crucial task is to engage in excellence in the programming side without cutting off the essential collaboration.

Australia in September

2010-08-05T09:19:00-04:00

Next month, I’ll be making the long journey down to Australia to spend ten days or so in Melbourne. Thus far I have a couple of talks booked. The main reason is to give a keynote at Agile Australia. I’ll also be speaking at Agile Melbourne. In both cases I’ll be doing a Suite of Talks, but I expect to pick mostly different topics for the two of them.

Continuous Delivery is Now Out

2010-07-30T18:01:00-04:00

Today I got my copy of Continuous Delivery, a book that I think is going to be probably the most important technical book of 2010. It talks about the build and deployment process: how to make it faster and less stressful. Almost every client ThoughtWorks has had in the last decade would have benefitted enormously from the techniques in this book, so I'm sure it has a huge potential to improve software development.

This week I've been working on the tutorial that Jez and I are doing at Agile 2010, and using my draft copy reinforced to me how useful a book this is. Jez and I hope to be doing a number of talks on this material over the next year or so - so keep an eye out for us. In particular we will be doing a full day tutorial at QCon San Francisco.

Bliki: UtilityVsStrategicDichotomy

2010-07-29T09:58:00-04:00

One of the steady themes I've seen throughout my career is that of the nature and importance of software development. Recently a prospect told one of our salespeople that "software is like sewage pipes, I want it to work reliably and I don't want to know about the details". This is the kind of approach that Nicholas Carr talked about in IT Doesn't Matter. On a contrasting note we've done work for many businesses where IT has been a clearer strategic enabler to their business, allowing them to enter new markets or significantly increase their market share. So is IT a utility, like sewage pipes, or a strategic asset?

I take that the view is that it can be either, depending on the system. A classic example of a utility IT project is payroll, everyone needs it, but it's something that most people want to 'just work'.

So what is the distinguishing factor between utility and strategic projects? To my mind it's all about whether the underlying business function is a differentiator or not. If how you do this function is a crucial part of what makes you better than the competition, then the software that supports this function needs to be as good as you can make it. Note that this distinction isn't about the software, but the business function. As Ross Pettit puts it "This is not a separation of IT by the nature of the technology, but into what technology does for the host business".

The most important point about this dichotomy is to realize that there are two kinds of software projects and they need to be treated entirely differently. The way you staff, run, and budget a strategic project is entirely different to how you do a utility project. Too often people assume that what is good for one is good for the other - and trouble inevitably follows.

Another consequence is that only a few projects are strategic. The 80/20 rule applies, except it may be more like 95/5. While certainly it's most common for people to not recognize the dichotomy at all, it's also common for people to think that too many projects are strategic.

One of the most important ways in which these efforts differ is where the risks lie. For utility projects the biggest risk is some kind of catastrophic error - you don't want the sewage pipe to break, or to miss payroll. So you need enough attention to make sure that doesn't happen, but other than that you want costs to be as low as possible. However with strategic projects, the biggest risk is not doing something before your competitors do. So you need to be able to react quickly. Cost is much less of an issue because the opportunity cost of not doing something is far greater than costs of software development itself.

This is not a static dichotomy. Business activities that are strategic can become a utility as time passes. Less often, a utility can become strategic if a company figures out how to make that activity a differentiator. (Apple did something like this with the design of personal computers.)

One way this dichotomy helps is in deciding between building custom software and installing a package. Since the definition of utility is that there's no differentiator, the obvious thing is to go with the package. For a strategic function you don't want the same software as your competitors because that would cripple your ability to differentiate.

Often people realize this and buy a package for a utility project, but then spend huge amounts of money customizing this - which is just as wasteful. My view is that for a utility function you buy the package and adjust your business process to match the software. Usually this is politically infeasible, so the workaround is to put a low grade software team to work on it. Provide enough care to avoid catastrophe, but otherwise you don't need a high-grade team.

Another way the dichotomy makes its influence felt is the role of agile methods. Most agilists tend to come from a strategic mindset, and the flexibility and rapid time-to-market that characterizes agile is crucial for strategic projects. For utility projects, however, the advantages of agile don't matter that much. I'm not sure whether using an agile approach for a utility project would be the wrong choice, but I am sure that it doesn't matter that much.

Like many classifications, there's a lot of grey in between. Yet this is one of those rare cases where I think there's a strong argument to turn up the contrast and force more binary thinking. As Ross commented in a discussion of a draft of this post: "'shades of grey' give license to pile things into the wrong category; things that are really utility will be given an inflated importance, rather than dispositioned as the utilities they really are." Forcing a binary decision, tilted to minimize what's in the strategic bucket, would help provide the focus that's often lacking in IT initiatives.

Ross goes so far as to argue that there shouldn't be a single IT department that's responsible for both utility and strategic work. The mindset and management attitudes that are needed for the two are just too different. It's like expecting the same people who design warehouses to design an arts museum.

To explore more...

Ross's article calls for a Glass-Stegall of IT departments.
Marc McNeil talks of projects as Tractors, nuclear power plants and the bleeding edge
Neal Ford points out that integration efforts, such as SOA, can't be strategic.
There is video of the talk I gave on this theme for AMP last year.

Pourquoi, pas comment

2010-07-15T18:24:00-04:00

A couple of weeks ago I was in Paris with Neal Ford to speak at USI 2010. Our talk looked at some aspects of why agile works. Rather than look at the techniques which is how agile does its thing, we looked more at some of the underlying forces - focusing on communication and feedback. The video is now available. (The introduction is in French but the talk is in English. Sadly my French is barely sufficient to order at a restaurant.)

Bliki: TeamRoom

2010-06-14T09:31:00-04:00

A common thing you find in agile projects is that the development team sits in a single open team room. It was advocated early on in Extreme Programming and called out as one of primary practices in the second edition. Agilists favor a open team room as it promotes lots of informal and deep communication between people on the team.

Why

Software development is an intense exercise in collaboration. An open space encourages regular conversations and interactions between people. You can see what everyone's doing and easily ask for help when you need to. Often you get serendipitous communication, where you overhear something that's really useful.

Hearing this, some people are concerned about noise, and would prefer private offices. In practice I find that teams don't find noise to be a serious issue. There's usually a hum of conversation going on, after all pair programming often accompanies this style of development. But the conversation isn't usually that distracting, particularly as you're focused on the conversation with your pair. I suspect the reason it's not that distracting is because the team has a common purpose around a collaborative activity. It isn't comparable to an open-plan office where everyone is doing something different.

Tips for a good Team Room

First make sure it is the right size for the team. While a team room should be open within itself, it should be closed to everyone else. In an ideal world you'd like flexible walls that can insulate one team from another, so an office consists of pods of teams. This is hard to do in practice. Our offices tend to be completely open, with little barriers between teams. This seems to work well-enough, although there are some complaints about the inter-team noise.

Pay attention to natural light. Humans are used to seeing the outside world, and all sorts of natural rhythms work off of light. So it's no surprise that people get very cranky if there isn't enough light about. I've spent plenty of days in enclosed conference rooms, and it certainly wears down my energy.

Provide enough space: about 50 square feet per person (4.5 square meters for those on metric).

With the right kind of space the next key thing is to ensure that the team has control over that space. An important part of agile thinking is that the team is responsible for how it works, and how it organizes its space is part of that. Ideally you want a team to have complete control over their space, with freedom to configure it how they like and reconfigure it at will. Things should be done so it's easy to move things around, because during the project the team will need to change things as the project changes.

The purpose-built tables in our Beijing office have handy baskets for power and other cables.

An immediate consequence of this is to ditch any kind of modular furniture that requires a facilities group to move more than an inch. Most teams I see use simple tables, and you can certainly go cheap here.

The biggest hassle is wires - primarily for power and network access. Ideally you want these underfloor or overhead so that people can easily route wires to the tables, wherever the tables happen to be.

The place to spend money on furniture is for good quality chairs. Programmers spend a lot of time sitting, and any physical damage due to poor posture will have a direct effect on the team's productivity - so don't stint here. It may be that some people will want strange chairs, such as balls or kneeling chairs. Do your best to accommodate them.

Some people are big fans of tables that can be adjusted between a sitting and standing height, as they find that standing for a while helps with back pain. These are harder to find, but worth looking into if your team members need it. Back pain is a common issue, but everyone's pain (and treatment) is different.

You'll need lots of wall space, as agilists love their information radiators. You want plenty of room for story walls, architectural diagrams, whatever people want to stick on the wall. A good bit of this wall space should be whiteboards so people can draw something whenever the mood takes them. Include some wheeled whiteboards. Make sure there's a digital camera around so people can easily record what's on the board. Now that displays are so cheap, consider getting some just for wall displays - this is particularly handy for dynamic displays such as build status. I've seen one team have projectors trained on a wall with various kinds of information displays.

The traditional layout is to have people working around clumps of desks. This gives you regular eye contact with the rest of your team. However I've heard many people sing the praises of the UPod.

People will occasionally need some private space, so ensure there are one or two small conference rooms available, with telephones. These can be used for privacy or when there's a concern about distraction. A large meeting room that the team can gather in for meetings away from the working space is also handy.

I've always been a big proponent of lots of monitor space. Clever software with multiple virtual workspaces is a great feature, but nothing is faster than just moving your eyes. Every workstation should have a couple of 20 inch monitors as a bare minimum. My desk has a pair of 20 inch monitors for my Ubuntu machine and a 25 inch for my mac laptop. I don't think that's at all excessive.

Software development is inherently creative, so expect to see lots of baubly distractions. Toys are commonly found around our teams (as Neal Ford says: "every team needs a plastic kangaroo"). There are good cognitive reasons why this is valuable, it's all about keeping the brain stimulated and creative.

Similarly do provide easy access to snacks and drinks. This helps support informal, conversational breaks in the team area. It's hard to be creative when you have to pay for awful coffee.

If you're working with remote workers, make it easy to set up a video link. Indeed many teams like having permanent video links to any remote workers so you can always maintain that casual video contact.

Useful Links

InfoQ summary of things for a team room.
William Pietri describes a team room he's used.
Bill Wake's gallery of team rooms and charts.
Rachel Davies's advice on building an agile environment.
Joel Spolsky is perhaps the best illustration for using individual offices rather than a single team room.
Rajeev Singh's thoughts on moving into this kind of team room at ThoughtWorks India.

My thanks to my ThoughtWorks colleagues for helping compile this information.

Craigslist Scam

2010-06-08T10:41:00-04:00

Over the weekend I’ve had a few emails from people in response to scams on craigslist. The scammer has been posting fake job adverts, using my name, for a driver or personal assistant. In case someone checks here I thought I should make it clear that I have never made any postings on craigslist and any postings you see there with my name are false.

Bliki: iPad

2010-06-04T13:49:00-04:00

I've not seen myself as an iFanboy. I didn't get an iPhone for ages after it came out, and only got one because it was the only way to upgrade my data plan to 3G. I use a mac, but I also have an Ubuntu desktop. But I have got an iPad, and I think it's a significant product.

I knew I wanted one right from the announcement. A while ago I used my desktop for work and kept my laptop floating about the house. I liked that arrangement as it allowed me to get away from my desk for reading the web and doing email. But now that I use Keynote, Aperture, and Omnigraffle I need to use my Mac with a big screen. So my laptop is stuck on my desktop while I'm at home. To compensate I bought a cheap laptop. But using a laptop like this was always kludgy. The clam shell is awkward when I'm reading rather than typing. The low battery life means I try to keep plugged in as much as I can. So a few weeks ago I splashed out.

The Good

The form factor and low weight makes it comfortable to use on a sofa.
There's enough battery life that I can forget about it as long as I charge it overnight.
The screen is gorgeous: bright, clear and sharp.
The touch UI is familiar (due to the iPhone) and works really well on a tablet device.
Reading books using the Kindle app works really well. I don't miss having a paper book and I appreciate the lighter weight.
Web surfing is mostly very good. I find myself using the news apps from BBC, New York Times and NPR a lot.
It's good for watching a movie. (The Netflix app could be really big here, although I haven't used it yet.) I appreciated the mix of reading and movie watching while flying back and forth to San Francisco in a middle seat; not to mention the fact that I used less than half my battery in the process.

The Bad

I worry about the lock-in when buying an electronic book. So far I'm thinking Amazon is less likely to lock me into a specific platform than iBooks. But I do worry that I'll have lots of books that I can't read any more some day.
Web-app sites, such as Google's, often don't work too well. For regular use they make good use of the difference between hover and click - and this doesn't work with the iPad. This pushed me to get an app for reading my news feeds, even though I like Google Reader.
Like with my Android phone and the iPhone, it's awkward to sync some text or html files from my other computers to my iPad. This is annoying as I like sharing information this way. Dropbox may be the answer, we'll see.
Hotels with restrictive wifi are a real pain. I want to work on my laptop with wifi and use that nice Netflix app on my iPad. Paying for wifi is bad enough, being unable to use it on multiple devices is awful.
While it's ok to look at my inbox, the mail app doesn't work well for threaded discussions on mailing lists.
I would really like to be able to change the font size in the web browser.

The Puzzling

Apple gets a lot of heat for being a curated software system, rather than the traditional open one. Given how much of my life I've lost to sorting out software incompatibilities I'm not sure whether to like curation of not. In any case the competition with Android and the like will prove interesting.

The Bottom Line

After a few hours with the iPad, I realized that I was feeling a sensation I'd only remembered once before: back in 1995 when someone showed me the World Wide Web. There's no dramatic new technology, we've seen tablets before just as we'd seen hypertext before. But the overall package was game-changer. I don't know if the iPad will be The Device, but I do think that this kind of device will make huge difference to how we read and watch things in the future. I certainly need to work at making sure what I produce in the future is built with these kinds of devices in mind.

Brazil Visit

2010-06-03T09:26:00-04:00

In a couple of weeks, I’ll be making my first trip to Brazil (and South America). I’ll arrive in Rio on Jun 13 and give a talk there on the 14th. After that I’ll head over to Porto Alegre and hang out at our office until Agile Brazil, where I’ll be doing a Keynote. My Rio and Agile Brazil talks will be similar - in that I’ll be doing a suite of talks, although I’ll probably not use the same suite in each place.

Continuous Delivery Book

2010-06-02T14:26:00-04:00

My signature series is actually getting two new books, as well as my DSL book, there is a probably more important book on Continuous Delivery. The book is written by two long-time friends and colleagues and talks about how to use deployment pipelines to get you that last mile from Continuous Integration to deployment

As well as the book I'll be doing some tutorials on Continuous Delivery with Jez Humble over the next few months. Our first one is at Agile 2010. You find more details about talks Jez is doing at continuousDelivery.com.

Final Draft of DSL book on RoughCut

2010-06-01T17:27:00-04:00

After sorting out the Final Draft for production, I rebuilt the rough-cut, which is now live on the site.

The biggest change with the content of the Final Draft is that I have three new pattern chapters for external DSLs from my colleague Rebecca Parsons: Regex Table Lexer, Recursive Descent Parser, and Parser Combinator. I’ve also reworked the “Hello Antlr” topic into the Parser Generator pattern.

DSL Book now at Final Draft

2010-05-26T14:18:00-04:00

I’ve now reached another significant step in my DSL book: Final Draft. This is the moment when I declare that I’m done with my content and release it to production. Copy-editing will begin soon, as well indexing, and various other sundry task to turn final text into a book ready to go to the printers.

It doesn’t mean that I’m completely done with the book. I will have to review the copy edits to ensure they are OK. I will also be working intimately with the production process, helping to improve the output for Safari Books Online and assisting the print production. I’ll also fix any errors spotted in the content, such as from people commenting on the rough-cut.

But the important step is that I’m done with the major thinking. I can’t do any big work on the content now unless it’s an emergency, so I’ll only be tidying up little things. I feel a sense of relief, a knowledge that it’s time to let the book go.

Bliki: CanonS90

2010-05-05T20:10:00-04:00

Like many obsessive snappers, I've recently got hold of the Canon S90 camera. It's small enough to fit in your pocket, but has the kind of things that people with pretensions to seriousness like: full manual controls, RAW file support, a good sensor, and an f2 lens.

I've had it for a few months now, and I'm really liking it. It's really small, fitting well into my pocket. Although I like using my DigitalSLR, there are plenty of times when I don't fancy the bulk and a good pocket camera is very handy. I did have a Panasonic Lumix TZ3 for that purpose, but the Canon offers significantly better features for me.

I'm very happy with the image quality, particularly in low light. The combination of reasonable high ISO performance and a stabilized f2 lens, is very potent. Even my DSLR can't give me stabilization and an f2 lens. True, I only get f2 at 28mm, f2.5 at 35mm, and then it falls off. But I can get some nice shots at 28 and 35mm that I wouldn't otherwise get.

The handling works well too. The S90 has got a lot of praise for its front control ring, that goes around the lens, as well as having a rear control ring. Although I confess that most of the time I leave it in program mode, I do like the ability to get the extra control when I need it.

The S90 has salved my micro 4/3 angst. I like small cameras, but I bought into the Canon DSLR range before the micro 4/3 were announced. That made me a bit sad, as I think the micro 4/3 would have been a good choice for me. But with the S90, I feel happy about the combo of DSLR and S90, which gives me a good choice between capability and bulk.

I don't see much in downsides. I agree with others that the rear control ring is a bit too easy to turn by accident. But on the whole it's a great camera, one that I carry with me a lot.

DSL Book now available as rough-cut

2010-04-26T16:01:00-04:00

My DSL book is now available as a rough-cut on safari books online. If you’re a member of safari you can see the book and make comments on it. If you’re not a member you can only see the TOC, preface, and first chapter.

I’m very keen to hear comments, so if you take a look do let me know what you think. I’m particularly interested in any errors I’ve made so I can fix them before we chop down the trees. Please, however, remember this is a rough-cut; in particular it hasn’t been through copy-editing. So expect speeling errors and mistakes grammatical - you don’t need to flag those as they should get picked up once the copy-editors are set on it.

I’m told I’m the very first author to produce the safari files myself using an automated book production process. Apart from the glow of vanity, a practical consequence of this is that I can (and will) generate updates as I put together useful updates. I’ll announce any updates here and on my twitter stream.

The cover of my Next Book

2010-04-23T09:37:00-04:00

Books in my signature series use photos of bridges, in homage to my wife's profession. This bridge is The Iron Bridge, the first bridge in the world to be made of iron, back in 1781. It's a symbol of the industrial revolution, which began near where I grew up in the Black Country. The photo is my own, which I took earlier this year. The design of the bridge looks somewhat unusual, nobody had built an iron bridge before, so they built it like they would build a wooden bridge. It took Thomas Telford to really use iron effectively in bridges.

Speaking in Atlanta and San Francisco Area

2010-04-21T19:33:00-04:00

I’ve got a couple of US speaking trips coming up, organized by ThoughtWorks. Next week (28-29 April) I’m off to Atlanta to give a couple of talks. Then in May (11-12) I’m in the San Francisco area to speak both down in the valley and in the city. As is my style these days, I’ll be doing a suite of talks on various topics that I’m currently interested in, so while I can’t say exactly what I’ll talk about, the talks will be the kinds of stuff I write about here. They’re all free events, so at least you have a good chance of getting value for money.

Bliki: Semat

2010-04-16T17:50:00-04:00

SEMAT (Software Engineering Method and Theory) is an effort initiated by Ivar Jacobson, Bertrand Meyer, and Richard Soley. Its stated aim is to "refound software engineering based on a solid theory, proven principles and best practices". Like many notorious people in the software world I was invited to participate. Thus far I've declined and feel the need to explain why.

The call for action seemed somewhat vague to me. Although it derided fads and fashions, it seemed very much like a fashion statement itself. To better understand what was going on I took a further look at some of the initial publications in Dr Dobb's Journal. From here I got the distinct impression that the central thrust of the initiative is to create a software meta-method-kernel - essentially a set of common process elements for software developments that you can rigorously compose into a method for your own project.

At this point I lost interest. I spent a fair bit of time in the 80's and 90's mucking around with this idea. In the end I decided that it was too hard and of limited value. Why this is so was primarily crystallized for me by Alistair Cockburn who explained that since people are the central element in software development, and people are inherently non-linear and unpredictable - such an effort is fundamentally doomed. Or at least it is until people become predictable agents that can be described with tractable mathematics - for which event I'm not holding my breath.

My views have since gone along the lines that software process is a much more multi-faceted thing than what a meta-method-kernel would usefully describe. Alistair's work on describing methods strikes me as a much more realistic approach.

Alistair actually did get involved in SEMAT and attended their inaugural meeting. His decision to withdraw further reinforces my lack of interest to participate myself.

Bliki: AmalfiCoast

2010-04-15T19:16:00-04:00

We've just returned from a week's vacation on the Amalfi Coast of Italy. For those contemplating a similar trip, here are a few scattered impressions.

It's a lovely part of the world. For a start it's in Italy, one of our favorite places to be on vacation: hiking, scenery, buildings, food, gardens, weather - all sorts of things we like. The Amalfi Coast also has that dramatic combination of mountains and sea that marks so many of my favorite places (Acadia, Côte d'Azur, Big Sur, Western Cape, West Coast of Scotland). Couple that with villages and farms clinging to the mountainside and it's all impossibly lovely.

On the way down from Rome we stopped at Caserta, which has a palace built to rival Versailles. We only took a look at the garden, which was a great example of formal French garden design. Well worth a couple of hours walking around.

We stayed for a couple of days at the Agristurismo Serafina. An agriturismo is a farm that provides accommodation. We were out of the way in the countryside and fed good local food. Some other Italian guests told us that Serafina was one of the earliest agriturismos and that many other agriturismos aren't farms, but really just hotels that have a few chickens or something to make them qualify. Here it was obvious that it was a working farm. The hosts didn't speak much English but were very friendly and helpful.

Were we to come back (which I certainly hope we shall) I'd consider not hiring a car. The local buses provide a good service, saving the hassle of driving and parking - the latter particularly awkward in the pretty coastal towns. The buses also make it easy to do one-way hikes.

The hiking was excellent. The trails we did were well marked, and the hosts at Serafina lent us some good maps. We also used the sunflower book that gave a good description of suitable walks.

The twisty, narrow roads, precipitous drops, and infamous Italian drivers make a very intimidating combination for driving. However I quickly got used to it as long as I got more laid-back, started to enjoy the antics of my fellow drivers, and lost my inhibitions about being on the wrong side of the road.

Ravello is a gem of a town with two lovely gardens. However I'm convinced that I'm neither rich enough nor beautiful enough to fit in there.

We spent a few days in Sorrento staying at the Hotel Mignon. It's the kind of hotel we like for vacation: simple, clean, friendly and right in the middle of town so we could stroll around the old town easily. Sorrento isn't as cute as places like Ravello or Amalfi, but makes a good base.

The boat trip around Capri is touristy but fun, the Blue Grotto extravagantly so. But once you get away from the boutiquey center there's some nice hikes.

Steps towards REST

2010-03-18T10:11:00-04:00

Last year Leonard Richardson gave a talk on QCon that included a maturity model for RESTful web services. The model is a good way to sneak up on understanding REST principles and the authors of REST in Practice are using it to help their discussion of how to use REST. Here’s my take on explaining the model, which I found helpful in my understanding of what makes REST tick:

Richardson Maturity Model: steps toward the glory of REST

Bliki: VcsSurvey

2010-03-08T14:02:00-05:00

When I discussed VersionControlTools I said that it was an unscientific agglomeration of opinion. As I was doing it I realized that I could add some spurious but mesmerizing numbers to my analysis by doing a survey. Google's spreadsheet makes the mechanics of conducting a survey really simple, so I couldn't resist.

I conducted the survey from February 23 2010 until March 3 2010 on the ThoughtWorks software development mailing list. I got 99 replies. In the survey I asked everyone to rate a number of version control tools using the following options:

Best in Class: Either the best VCS or equal best
OK: Not the best, but you're OK with it.
Problematic: You would argue that the team really ought to be using something else
Dangerous: This tool is really bad and ThoughtWorks should press hard to have it changed
No opinion: You haven't used it

The results were this:

Tool	Best	OK	Problematic	Dangerous	No Opinion	Active Responses	Approval %
Subversion	20	72	6	1	0	99	93%
git	65	19	1	0	14	85	99%
Mercurial	33	27	2	0	36	62	97%
ClearCase	0	3	14	41	41	58	5%
TFS	0	0	32	22	44	54	0%
CVS	0	14	59	11	15	84	17%
Bazaar	1	13	3	0	80	17	82%
Perforce	1	26	16	1	54	44	61%
VSS	1	1	11	64	22	77	3%

As well as the raw summary values, I've added two calculated columns here to help summarize the results.

Active Responses: The total of responses excluding "No Opinion". (eg for git: 65 + 19 + 1 + 0)
Approval %: The sum of best and ok responses divided by active responses, expressed as a percentage. (eg for git: (65 + 19) / 85)

The graph shows a scatter plot of approval percentage and active responses. As you can see there's a clear cluster around Subversion, git, and Mercurial with high approval and a large amount of responses. It's also clear that there's a big divide in approval between those three, together with Bazaar and Perforce, versus the rest.

Although the graph captures the headline information well, there's a couple of other subtleties I should mention.

Although the trio of Subversion, git, and Mercurial cluster close together on approval, git does get a notably higher amount of best scores: (65 versus 20 and 33).
VSS got the most "dangerous" responses, but a couple of people approved of it.
Neither TFS or ClearCase are liked much, but ClearCase got more "dangerous" responses than TFS (41 versus 22).
Don't read too much into small differences as I'm sure they aren't significant. I'm sure the difference in approval percentage between VSS, TFS, and ClearCase isn't signifcant, but the difference between these three and the leaders is.

Some caveats. This is a survey of opinion of ThoughtWorkers who follow our internal software development discussion list, nothing more. It's possible some of them may have been biased by my previous article (although unlikely, since I've never managed to get my ThoughtBot opinion-control software to work reliably). Opinions of tools are often colored by processes that are more about the organization than the tool itself. But despite these, I think it's an interesting data point.

I should also stress the important point to take away from this isn't the comparison between those close in the numbers, eg comparing git and Mercurial or comparing TFS and ClearCase. Any survey like this has a certain amount of noise in it, and I suspect the noise here is greater than such a difference. The important point is the big approval gap between the leading tools (Subversion, git, and Mercurial) and the laggards - essentially the point in VersionControlTools.

Bliki: ToyotaFailings

2010-03-02T19:45:00-05:00

One of the arguments used to support the adoption of lean techniques in software is the success of Toyota. So do Toyota's recent quality failings undermine the case for lean software development?

One answer for this is to take a sense of proportion. Lean manufacturing techniques were the underpinning of Toyota's rise from an insignificant company in the 1950's to a global giant in the 2000's. By the 1990's other car companies, and many other manufacturers, were busily copying Toyota's techniques. The general sense is that copying these techniques did much to raise the overall quality of cars in the last decade or so. I would be very surprised if the recent problems at Toyota are enough negate that half-century of success.

But a better answer is to remember that Lean manufacturing is about manufacturing not software. The application of lean ideas to software development is a consequence of MetaphoricQuestioning. Lean ideas can help us come up with better ideas for software development, and as such are valuable. But in the end their usefulness lies with how they are used in software and they should be judged on their record here. Their history in manufacturing, both good and bad, is another industry.

Bliki: BlueGreenDeployment

2010-03-01T09:25:00-05:00

One of the goals that my colleagues and I urge on our clients is that of a completely automated deployment process. Automating your deployment helps reduce the frictions and delays that crop up in between getting the software "done" and getting it to realize its value. Dave Farley and Jez Humble are finishing up a book on this topic - Continuous Delivery. It builds upon many of the ideas that are commonly associated with Continuous Integration, driving more towards this ability to rapidly put software into production and get it doing something. Their section on blue-green deployment caught my eye as one of those techniques that's underused, so I thought I'd give a brief overview of it here.

One of the challenges with automating deployment is the cut-over itself, taking software from the final stage of testing to live production. You usually need to do this quickly in order to minimize downtime. The blue-green deployment approach does this by ensuring you have two production environments, as identical as possible. At any time one of them, let's say blue for the example, is live. As you prepare a new release of your software you do your final stage of testing in the green environment. Once the software is working in the green environment, you switch the router so that all incoming requests go to the green environment - the blue one is now idle.

Blue-green deployment also gives you a rapid way to rollback - if anything goes wrong you switch the router back to your blue environment. There's still the issue of dealing with missed transactions while the green environment was live, but depending on your design you may be able to feed transactions to both environments in such a way as to keep the blue environment as a backup when the green is live. Or you may be able to put the application in read-only mode before cut-over, run it for a while in read-only mode, and then switch it to read-write mode. That may be enough to flush out many outstanding issues.

The two environments need to be different but as identical as possible. In some situations they can be different pieces of hardware, or they can be different virtual machines running on the same (or different) hardware. They can also be a single operating environment partitioned into separate zones with separate IP addresses for the two slices.

An advantage of this approach is that it's the same basic mechanism as you need to get a hot-standby working. Hence this allows you to test your disaster-recovery procedure on every release. (I hope that you release more frequently than you have a disaster.)

The fundamental idea is to have two easily switchable environments to switch between, there are plenty of ways to vary the details. One project did the switch by bouncing the web server rather than working on the router. Another variation would be to use the same database, making the blue-green switches for web and domain layers.

This technique has been "out there" for ages, but I don't see it used as often as it should be. Some foggy combination of Dan North and Jez Humble came up with the name.

"IT - more than Tools and Technology" track at QCon London

2010-02-24T12:34:00-05:00

I’ve been a regular speaker at the QCon and JAOO conferences over the last few years. At QCon London this year, I’m involved in a somewhat off-beat track of talks. Software Developers have often had a habit of focusing primarily on the mechanics of developing software well, and not thinking much about the societal value of that software. This is a common strand in many professions, but as the influence of software development grows, I feel it’s increasingly important that we get more engaged in the consequences of the software we build.

This track is a chance to explore some of these issues. As well as myself, there are talks about some work done with UNICEF to make effective use technology in much poorer parts of the world, the role of IT in reducing carbon footprint, and the contribution of team diversity to innovation.

Texas Speaking Events Rescheduled

2010-02-06T12:34:00-05:00

The family medical issue has been resolved happily, so I’m free to go back on the road. We’ve thus rescheduled the events I was supposed to do last month in Texas.

On February 23rd I’ll be speaking at DFW Scrum in Dallas.
On February 25th ThoughtWorks is organizing a technology forum in Austin.

As is usual for me, I haven’t planned exactly what I’ll talk about yet, but it’ll revolve around my usual topics of software design and agile methods.

Apologies for Canceling Texas Speaking Events

2010-01-15T20:00:00-05:00

I’m afraid I’ve had to cancel my speaking events in Dallas and Austin next week due to a family medical problem. As I write this, it’s not clear how serious the problem is going to be, but there is a good chance that I won’t be able to travel to Texas next week. As a result we felt it was best to cancel the events, while we still have a few days notice. We do intent to reschedule as soon the as dust settles. My Texas ThoughtWorkers are very keen to have me come out and do these talks, so we want to do them as soon as we reasonably can.

My apologies for this, and I hope you understand. In particular I want to thank the various collaborators in organizing these events for being very understanding under the awkward circumstances.