Agile Software Development
In the last decade agile software development has moved from being a cult technique to an increasing part of the mainstream. I was lucky enough to be at the beginning of this story, with early experiences on the 'birth project' of Extreme Programming and a co-author of the Manifesto for Agile Software Development. ThoughtWorks started using agile techniques in 2000 and we've since successfully used them on our projects world-wide. We've learned a huge amount about using agile methods in enterprise settings and are committed to sharing this learning to help foster their intelligent adoption.
The Essence of Agile Software
It's been over a decade since the developers of agile methods first started to talk about their approaches. In this time agile thinking has changed from a niche activity to an approach that is widely used. However, like any popular technique, agile software development has suffered from semantic diffusion, so much of what we see under the name of agile doesn't bear much resemblance to what the early pioneers were doing. So I think it's important to revisit the essential elements of agile thinking.
I've always seen the essence of agile thinking resting on two contrasts with traditional plan-driven software engineering.
- is adaptive rather than predictive
- is people-oriented rather than process-oriented
Plan-driven engineering expects us to come up with a predictive plan that precedes development. The plan lays out the people, resources and timelines for the overall project. Software design is also done up-front, with implementation expected to conform with this design. Success is measured according to how well development follows this plan.
Agile plans are a baseline that we use to help us control change. Agile teams plan just as carefully as traditional teams, but the plans are constantly changing to reflect the things we learn during a project. Success is based on value delivered by the software.
Plan-driven engineering seeks a process which provides enough structure to reduce individual variations to insignificance. Such an industrial process is more predictable, copes better when people transfer, and is easier to define skills and career paths.
Agile engineering sees software development as a primarily human activity, where the people involved and how they bond as a team are the primary driver behind success. Processes (and tools) can enhance a team's effectiveness, but are always second-order influences.
I explored these contrasts in more detail in my essay The New Methodology which I originally wrote in 2000 and updated in 2005. I continue to feel it best expresses how I think about the essence of agile software development.
It may not have started it all, but the manifesto gave the movement a name together with a dollop of initial energy. A decade later it still captures the essence of what agile methods are about.
In this twenty minute talk I introduce the essence of agile software development and the agile fluency model
Adopting agile software development is neither a quick nor easy path. Diana Larsen and James Shore (experienced agile coaches) have come up with a way to think about how teams progress through stages of fluency in agile thinking. The path begins by focusing on value and then progresses to delivering and optimizing that value. Diana and James outline the benefits of each stage and the what investments you need to make to get there.
To make agile work, you need solid technical practices. A lot of agile education under-emphasizes these, but if you skimp on this you won't gain the productivity and responsiveness benefits that agile development can give you (stranding you at level 1 of the agile fluency model.) This is one of the reasons that I still think that Extreme Programming is the most valuable of the named agile methods as a core and starting point.
From the customers' point of view, the essential benefit of the agile approach is that it allows software development to be a transparent process. Software grows in a visible way, so that businesses can learn from previous iterations when considering future work.
In order to make this work fully you need a deployment pipeline that ensures that software is built in small production-ready increments. Many of the dominant internet organizations release these increments, performing many production deployments every day. This allows a whole new relationship between the developers of software and their users and customers.
From the very earliest days of agile methods, people have asked what role there is for architectural or design thinking. A common misconception is that since agile methods drop the notion of a detailed up-front design artifact, that there is no room for architecture in an agile project. In my keynote at the first-ever agile conference, I pointed out that design was every bit as important for agile projects, but it manifests itself differently, becoming an evolutionary approach.
Recently my colleague Molly Dishman and I have given a couple of talks about agile + architecture: what architecture is, and how you ensure it's happening on an agile project.
Agile methods are founded upon making change easy, this means that you need to have a code base that's amenable to change and the discipline to make changes safely. Refactoring is the foundational technique to change and improve the design of an existing code base. While my book on refactoring was written a decade ago, the techniques it teaches are just the same today.
Improving human collaboration is at the heart of agile thinking. Communication and feedback are two of the stated values of Extreme Programming, and agilists look to find ways to maximise these as part of their projects
A common practice in agile projects is the short daily team meeting - called a stand-up meeting or scrum. Done well, these help team members coordinate their activities effectively, adding energy to the team. Done badly, they are a boring and empty ritual that sucks the life out of developing software.
Jason Yip's advice on doing stand-ups has become the primary article on making the daily meeting work well.
When we were looking into opening an office in India in 2002, people doubted that agile methods could be used in an offshore development project. I spent time talking to ThoughtWorkers in India and the US to produce this article on how we did it.
Dan North and I think that most important factor in software development is the communication between users and developers. In a keynote for QCon London, we explore this relationship, saying that a bridge is better than ferrymen.
Many programmers have a poor opinion of metrics, having seen them lead to problems with development teams. Pat Kua looks at how these efforts go wrong and describes some guidelines for how they can be used effectively: linking directly goals, using short tracking periods, focusing on trends rather than absolutes, and dropping them when they are no longer driving change.
Pat Kua has also written two very useful books for agile teams
After Kent wrote the initial "white book" on Extreme Programming, he collaborated with me on writing a book concentrating on the planning process in Extreme Programming. While other agile planning books have appeared since, I still like this book for its brief coverage of the essence of planning in XP.
To explore further…
…take a look at the tags:
The banner photo at the top of this page (and most agile-oriented pages on this site) comes from the very first agile conference.