Videos

Most of what you’ll find on this site is writing, but I know that many people enjoy a video experience. I haven’t got into video production, it’s difficult work and not something I find worthwhile. But I do give talks, and often these talks are now captured on video. So I’ve put together this page to pull together all the talks and other video material I’ve been involved in.

I do repeat talks, so several talks have multiple video versions to choose from. I’ve also put useful links on this page to help you explore further than the talk itself.

Most of my talks are conference keynotes, and for the last decade or two I’ve been doing keynotes under the title Software Development in the 21st Century. The title is deliberately vague, allowing me a pretty free rein to talk about whatever I fancy on the day. In recent years, I’ve structured these keynotes of Suites of Talks, doing two or three twenty minute talks in the keynote slot. As these get the video treatment, I’ve encouraged conferences to break up the video and release the individual talks separately rather than bundled into the whole suite. For this page I’ve described these short talks separately. Not all videos separate these talk segments, so for those that combined them I’ve linked into the middle of the video to get as close as the video allows me to the start of the actual talk segment (these are marked with “✂”)

The Art of Agile Software

Essence, fluency, reflections since the manifesto, the yawning crevasse of Doom

Software Architecture

Agile architecture, microservices

Software Development in the 21st Century

All the segments I use in my keynotes

The Art of Agile Software

"How many people in this room have been on a software project where there's been significant changes in requirements during the course of the project?"

The art of developing software is one we’re learning as we do it. I’ve been mostly interested in software systems with to support needs that are both important and uncertain. This led to my interest in agile methods, and I was an early pioneer with Extreme Programming.

Agile Essence and Fluency

The essential elements of agile software development and how you gain fluency as you learn

expand…

It's been over a decade since we wrote the Manifesto for Agile Software Development, and the agile meme has been more successful than we ever could have hoped for. But like any success, there is the regular danger of Semantic Diffusion. I try to combat this disease by describing the essence of agile software development: preferring adaptive planning to predictive planning and favoring people over process.

I then describe the Agile Fluency model, which I find an effective way to think about how agile teams become proficient, and the steps you typically go through as you become a more skillful user of agile techniques.

Further Reading

Why Agile Software Works

with Neal Ford

Why is it that agile approaches work so effectively?

expand…

Neal Ford and I gave a talk at USI in Paris (2010) on some aspects of why agile works (as opposed to how). This probes at some of the core forces that makes agile effective, rather than looking at techniques. In particular we look at role of communication and feedback and how they interplay in agile environments.

Sadly the video appears to be truncated and cuts off in the middle of the talk. I haven't been able to figure out how to get the full video posted.

Agile Manifesto: 10 years later

expand…

I gave this talk ten years after we wrote the agile manifesto. I set out the historical context in which we wrote the manifesto, explain why the semantic diffusion we've experienced since is an inevitable consequence of its success, argue that those who say they don't care about agile are usually wrong, and highlight two areas where we see some interesting new activity in agile thinking.

Further Reading

Retake on the Agile Manifesto

with Dave Thomas, Jez Humble, Katherine Kirk, & Tatiana Badiceanu

Should we kill agile software development?

expand…

In 2014 Dave "Pragmatic" Thomas got irritated with the state of the agile software world saying Agile is Dead. The organizers of the goto conference in Aarhus, who have long been active explorers of agile styles felt that was a good opportunity to bring together him and myself, as two of the authors of the manifesto, together with some people who have been on the receiving end - using and extending agile approaches in the field.

Further Reading

The Yawning Crevasse of Doom"

with Dan North

The most important factor in software development is the communication between users and developers

expand…

A keynote for QCon 2007 that I did with my colleague Dan North. We both see the gap between developers and their customers/users as the biggest problem in software development. (We'd call it a chasm, but that word is so overused.) Here we talk about this gap, why it's important, and what we need to do to cross it. In particular we argue that the traditional role of an intermediary Business Analyst acts as a ferry, while what we really need is a bridge that enables directly contact between developers and their customers (and analysts can build and maintain that bridge). This is one of my favorite joint keynotes, both because I think the topic is so important, and because Dan is such stimulating co-speaker.

Dan North helped me explain why bridges are better than ferrymen

Software Architecture

"Architecture is the important stuff, whatever that happens to be"

Since the begining of agile methods, there's always been a deep debate on what role (if any) software architecture should play on agile projects. Much of this depends on what you consider architecture should be.

Making Architecture Matter

What architecture is and why it matters

expand…

I was asked to do a fourteen minute minute keynote at OSCON to explain why architecture was important. I decided the best thing was to begin by exploring the meaning of that awkward term, guided by my favorite mailing list post from Ralph Johnson. Once I'd done that I moved onto why it matters by focusing on the economic argument of the Design Stamina Hypothesis.

Further Reading

Agile Architecture

with Molly Dishman

What is architecture, why is it important, and how do we ensure it happens?

expand…

Software architecture is an ill-defined concept, that makes an inappropriate terminological borrowing from the building industry. We consider architecture to be the selection of the most important attributes of a system, with a focus on those things that are hard to change. Architecture is something that can evolve as a system grows, but only by putting effort and attention to ensuring it's looked after. We can do that through a combination of initial visioning and on-going efforts.

(Dallas video includes Q&A leading to 65 minutes total.)

Economics of Software Design

The point of spending effort on design is to improve productivity - delivering features quickly

expand…

Often people justify effort on software design by pointing to the need for greater craftsmanship and quality. My view is that such moralizing arguments are wrong, that instead we should focus on the economics. Most software efforts slow over time, as the weight of bad design decisions slow down our teams. Attention to design can reduce or even reverse this.

I've found that the metaphor of Technical Debt is a good way to think about the consequences of bad design - do we pay interest or pay off the principal. Some argue that technical debt isn't the result of sloppy design, but I point out that technical debt comes from various causes, and even the best teams will generate some.

Further Reading

Rebecca Parsons and Molly Dishman have shared the stage with me to talk about agility and architecture

Agilists and Architects: Allies not Adversaries

with Rebecca Parsons

Architects should play an important, if different, role in agile projects.

expand…

At QCon San Francisco 2008 Rebecca Parsons and I gave a talk about how agile approaches work with enterprise architecture groups. At the moment there's a lot of distrust and conflict between agile project teams and architecture groups. We dig into why this is so, and explore ways that these groups can work together.

A Conversation about Hexagonal Rails

with Badri Janakiraman

Hexagonal architecture, choosing how to interact with your database, and how to design with frameworks like Ruby on Rails

expand…

A discussion between me and Badri Janakiraman- one of ThoughtWorks's most senior developers about the architecture of Rails applications. We start by talking about the notion of Hexagonal Architecture and the role of the database in enterprise applications, particularly Ruby on Rails app. These principles colors the decision between using the Active Record or Data Mapper patterns to organize collaboration with the database. We then move onto talking about how to work with a full-stack application framework like Rails, with the choice between treating it as a platform or as a suite of components.

Further Reading

Continuous Delivery

Build software so you can always deploy your current code, reducing risk and getting faster feedback

expand…

Continuous Delivery is now becoming a central practice for effective software delivery organizations. This talk explains the essentials of how it works, the role of a deployment pipeline, the difference between continuous delivery and continuous deployment, and some vital ingredients. It also covers the three main benefits of Continuous Delivery: reducing deployment risk, believable progress, and user feedback.

Further Reading

Architecture without Architects

with Erik Dörnenburg

Architecture is both important, and something that doesn't need traditional software architects

expand…

The title software architect comes with many connotations, and often these are not good. Developers think of hand-waivers who inhabit ivory towers and have forgotten how to write code. Project managers think of technologists who are chasing perfection in initiatives that are serving obscure technical purposes. Yet, for the success of any software project architecture is crucial, particularly with the current interest in microservice architectures.

We argue that we can support good architecture without a traditional architecture role, introducing techniques to get good designs and sustainable applications.

(micro)Services and Architecture

Microservices

Microservices turned out to be the hot software architecture of 2014

expand…

A 20 minute introductory talk on microservices. I cover the definition of microservices, compare it to a more monolithic approach, and outline important things you have to get right before you should deploy a microservice application.

Further Reading

Does my bus look big in this?

with Jim Webber

We take an irreverently critical look at the SOA mainstream and suggest an alternative approach

expand…

My colleague Jim Webber has built quite a reputation for taking a lightweight and business-oriented approach to integration in the enterprise. He also has a reputation for being a very robust and entertaining speaker. So I was as nervous as I was excited to share the stage with him for a keynote at QCon 2008. He put together a wonderfully funny presentation with some serious tidbits of meat woven through it. We then just dove in and did it - possibly helped by the pre-talk pint. We talk about the history of Enterprise Integration, the growth of systems that think they are strong but are really just fat, the role of agile thinking, the influence of the web (including Jim's unique theory on why it was invented), and how this leads to Guerilla SOA.

Infrastructure as Code

Define your infrastructure configuration with executable code

expand…

I grew up in the Iron Age, where new servers had to be ordered as physical machines, but now we live in the Cloud Age where new servers can be spun up on demand in minutes. To take advantage of the speed and flexibility of the cloud age, we have to rethink how we manage infrastructure.

Infrastructure as Code keeps infrastructure definitions in an executable form, which can then be managed like any other code artifact and kept in version control. This provides more accurate documentation, and infrastructure that can be subject to the same build and test disciplines as application code. This allows us to scale to larger infrastructure configurations while maintaining a greater degree of consistency, reduces the risk of change, and allows us to rapidly support new demand.

Further Reading

The Many Meanings of Event-Driven Architecture

Throughout my career I've come accross architectures which are described as "event-driven". But I've found that this phrase means many different things, which I've boiled down into some combination of four patterns.

expand…

Late in 2016, I attended an architecture summit of senior ThoughtWorks developers to explore the various work we'd done under the heading of "event-driven". We confirmed that this phrase led to quite different things, which often got confused together. Instead we found it helpful to focus on four patterns, which we could define with more precision:

  • Event notification: components communicating via events
  • Event-based State Transfer: allowing components to access data without calling the source.
  • Event Sourcing: using an event log as the primary record for a system
  • CQRS: having a separate component for updating a store from any readers of the store

Further Reading

Is TDD Dead?

David Heinemeier Hansson gave a provocative talk at RailsConf in 2014 which led to a series of Hangouts as he, Kent Beck, and I discussed the role of TDD in Software Development.

Hangout

2014

5 videos adding up to 3¼ hours

 

The changing face of Data

Introduction to NoSQL

An introduction to NoSQL databases, covering the types of databases, consistency issues, and the role they play in data storage.

expand…

The origins of the term "NoSQL" were a hashtag for a twitter meetup, but they turned into the most serious challenge to the twenty year hegemony of relational databases. Due to the accidental nature of their name, they cover a wide range of with not much definition - but it's useful to group many of them under the moniker of Aggregate-Oriented Databases.

NoSQL databases introduce raise issues around consistency, but it's worthwhile remembering that even with ACID transactions we still often have to manage concurrent updates in our applications. The ability of many NoSQL databases to support distributed data further complicates consistency, leading to the CAP theorem's trade-off between consistency and availability (and response time). This trade-off is fundamentally a business decision, not a technical one.

NoSQL databases are a serious choice for modern data needs, but not the only choice. We are now in a period of Polyglot Persistence where we have to choose our data storage technology based on our specific data access needs.

This is my most popular talk (with over 300,000 views for the original goto Aarhus video).

Further Reading

What is NoSQL and is it the future of databases?

NoSQL and Consistency

How do NoSQL databases change how we have to think about database consistency?

expand…

Most NoSQL databases force people to think differently about consistency than you find in the relational world. Aggregate-oriented databases naturally remove some of need for transactions in relational systems. Database transactions do not prevent us from having to deal with the problems in concurrent updates. Adding distribution to our data increases the kind of consistency issues we need to deal with. The CAP theorem is primarily about the trade-off between consistency and availability (indeed latency) in a distributed system - a trade-off that is primarily a business decision.

(This talk is the consistency part of my Introduction to NoSQL talk, and duplicates material from that talk.)

Further Reading

The Evolving Panorama of Data

with Rebecca Parsons

expand…

Our keynote at QCon London 2012 looks at the role data is playing in our lives (and that it's doing more than just getting bigger). We start by looking at how the world of data is changing: its growing, becoming more distributed and connected. We then move to the industry's response: the rise of NoSQL, the shift to service integration, the appearance of event sourcing, the impact of clouds and new analytics with a greater role for visualization. We take a quick look at how data is being used now, with a particular emphasis from Rebecca on data in the developing world. Finally we consider what does all this mean to our personal responsibilities as software professionals.

Further Reading

Schemaless

Usually when people say a data structure is schemaless, they're wrong. There's a schema, it's just an implicit schema.

expand…

There's much talk about schemaless databases these days, but there is almost always a schema. An implicit schema seems flexible, but is usually worse because it makes harder to figure out how to use data. When people want schemalessness, what they usually need is Variable State, which is useful for custom fields and non-uniform data structures.

Further Reading

Event Sourcing

Treat all data the way we use version-control

expand…

Event Sourcing is an approach which handles updates by storing an event that describes the update and then processing that event to change the current application state. The event log then becomes the authoritative store of information, allowing you to delete any application states and rebuild them from the event store. Essentially this is the approach taken by version-control systems. Using event sourcing opens up several advantages in audit, ability to query historic states, debugging, and distribution.

Further Reading

  • Article My 2005 article explaining the technique in more detail

What impact should software development have on the world?

Not Just Code Monkeys

My biggest problem with agile software development, and the questions that flow from it.

expand…

This is a tricky talk to describe. Usually I like a title and abstract to describe what the talk is about - but this talk is a journey, and I don't want to tell you where I'm going, but instead to explore the ground with me. I will say that it starts with my biggest problem with most adoption of agile software development - the nature of the interaction between users, analysts, and programmers. It goes on to explore these roles, raising questions about the relationship of programmers to users, our responsibilities to them, and finally the Two Great Challenges that I think programmers need to face up to.

Our Responsibility to Defeat Mass Surveillance

with Erik Dörnenburg

Software developers have a duty to preserve privacy on the internet

expand…

Software professionals cannot consider ourselves to be merely order takers who do whatever our funders require, we are responsible for how our software affects our users and the broader society. Even if we think we have nothing to hide, our privacy is necessary to protect the bothersome people who prevent corruption and allow society to progress. The movement of email to online services has led to worrying concentration of email provision, which makes it easier to carry out mass surveillance of an important form of our communications. Even seemingly innocuous interception can lead to serious problems because this information about us is valuable to corporations, even if innocuous to governments.

We need to reduce these problems by working to widen the use of encryption for email, so that the cost of mass surveillance becomes prohibitive. The challenge for this is primarily a challenge of user-experience and software packaging, not something that requires great understanding of cryptography.

(The first 12 minutes of this talk is a compressed version of my Not Just Code Monkeys talk.)

Further Reading

Over the years I've given talks with Erik Dörnenburg about software architecture, TDD, and now the important role we developers have to play in maintaining internet privacy.

Interview: Privacy on the Internet

with Erik Dörnenburg, Ola Bini, & Tim Bray

Software Design in the 21st Century

Here are all the talk segments that I've used in my "Software Design in the 21st Century" keynotes

Agile Essence and Fluency

The essential elements of agile software development and how you gain fluency as you learn

expand…

It's been over a decade since we wrote the Manifesto for Agile Software Development, and the agile meme has been more successful than we ever could have hoped for. But like any success, there is the regular danger of Semantic Diffusion. I try to combat this disease by describing the essence of agile software development: preferring adaptive planning to predictive planning and favoring people over process.

I then describe the Agile Fluency model, which I find an effective way to think about how agile teams become proficient, and the steps you typically go through as you become a more skillful user of agile techniques.

Further Reading

Continuous Delivery

Build software so you can always deploy your current code, reducing risk and getting faster feedback

expand…

Continuous Delivery is now becoming a central practice for effective software delivery organizations. This talk explains the essentials of how it works, the role of a deployment pipeline, the difference between continuous delivery and continuous deployment, and some vital ingredients. It also covers the three main benefits of Continuous Delivery: reducing deployment risk, believable progress, and user feedback.

Further Reading

Economics of Software Design

The point of spending effort on design is to improve productivity - delivering features quickly

expand…

Often people justify effort on software design by pointing to the need for greater craftsmanship and quality. My view is that such moralizing arguments are wrong, that instead we should focus on the economics. Most software efforts slow over time, as the weight of bad design decisions slow down our teams. Attention to design can reduce or even reverse this.

I've found that the metaphor of Technical Debt is a good way to think about the consequences of bad design - do we pay interest or pay off the principal. Some argue that technical debt isn't the result of sloppy design, but I point out that technical debt comes from various causes, and even the best teams will generate some.

Further Reading

Event Sourcing

Treat all data the way we use version-control

expand…

Event Sourcing is an approach which handles updates by storing an event that describes the update and then processing that event to change the current application state. The event log then becomes the authoritative store of information, allowing you to delete any application states and rebuild them from the event store. Essentially this is the approach taken by version-control systems. Using event sourcing opens up several advantages in audit, ability to query historic states, debugging, and distribution.

Further Reading

  • Article My 2005 article explaining the technique in more detail

Infrastructure as Code

Define your infrastructure configuration with executable code

expand…

I grew up in the Iron Age, where new servers had to be ordered as physical machines, but now we live in the Cloud Age where new servers can be spun up on demand in minutes. To take advantage of the speed and flexibility of the cloud age, we have to rethink how we manage infrastructure.

Infrastructure as Code keeps infrastructure definitions in an executable form, which can then be managed like any other code artifact and kept in version control. This provides more accurate documentation, and infrastructure that can be subject to the same build and test disciplines as application code. This allows us to scale to larger infrastructure configurations while maintaining a greater degree of consistency, reduces the risk of change, and allows us to rapidly support new demand.

Further Reading

Microservices

Microservices turned out to be the hot software architecture of 2014

expand…

A 20 minute introductory talk on microservices. I cover the definition of microservices, compare it to a more monolithic approach, and outline important things you have to get right before you should deploy a microservice application.

Further Reading

NoSQL and Consistency

How do NoSQL databases change how we have to think about database consistency?

expand…

Most NoSQL databases force people to think differently about consistency than you find in the relational world. Aggregate-oriented databases naturally remove some of need for transactions in relational systems. Database transactions do not prevent us from having to deal with the problems in concurrent updates. Adding distribution to our data increases the kind of consistency issues we need to deal with. The CAP theorem is primarily about the trade-off between consistency and availability (indeed latency) in a distributed system - a trade-off that is primarily a business decision.

(This talk is the consistency part of my Introduction to NoSQL talk, and duplicates material from that talk.)

Further Reading

Non-Determinism and Testing

Non-deterministic tests are a disease that can destroy all the value in your testing.

expand…

Non-deterministic tests are tests that sometimes pass, sometimes fail, without any change in the underlying code. Left untreated, they make your whole test suite useless. First you need to quarantine them, and then fix them. Common causes include lack of test isolation, asynchrony, and talking to remote services

Further Reading

Not Just Code Monkeys

My biggest problem with agile software development, and the questions that flow from it.

expand…

This is a tricky talk to describe. Usually I like a title and abstract to describe what the talk is about - but this talk is a journey, and I don't want to tell you where I'm going, but instead to explore the ground with me. I will say that it starts with my biggest problem with most adoption of agile software development - the nature of the interaction between users, analysts, and programmers. It goes on to explore these roles, raising questions about the relationship of programmers to users, our responsibilities to them, and finally the Two Great Challenges that I think programmers need to face up to.

Schemaless

Usually when people say a data structure is schemaless, they're wrong. There's a schema, it's just an implicit schema.

expand…

There's much talk about schemaless databases these days, but there is almost always a schema. An implicit schema seems flexible, but is usually worse because it makes harder to figure out how to use data. When people want schemalessness, what they usually need is Variable State, which is useful for custom fields and non-uniform data structures.

Further Reading

Workflows of Refactoring

expand…

Refactoring is a vital technique to preserve the health of a code base. Often it's described in the context of Test-Driven Development, but there many places in a developer's workflow that refactoring can be used. It's particularly important to use it opportunistically, so that problems can be fixed as soon as they become apparent. The value of regular refactoring lies not in some call to craftsmanship, but for business reasons to ensure the future flow of valuable software.

Further Reading

And the rest…

3.years.of(:ruby)

expand…

For a talk at QCon London 2009 I surveyed ThoughtWorks use of Ruby from 2006-2008 in which time we did 41 projects. My talk covers our views on Ruby's producitivity, speed, and maintainability. I conclude that Ruby should be taken seriously as a development environment.

Further Reading

An Introduction to Language-Oriented Programming

An early introduction to using Domain-Specific Languages

expand…

Further Reading

Continuous Delivery (YOW 2011)

with Jez Humble

expand…

We give a one-hour overview of Continuous Delivery. Topics include the justification of Continuous Delivery, the deployment pipeline, continuous integration, devops, and deployment strategies. The highlight is Jez's personification of a release candidate as a hero in a greek myth.

Further Reading

Evolving a Mobile Implementation Strategy

with Giles Alexander

expand…

Mobile is still a smaller part of traffic than the traditional web, but its share is growing, so we need to think about our strategy for developing effective mobile applications. We discuss thinking about a product vision, separating the styles of user engagement into "Lean-forward", "Lean-back", and "Look-down" styles; while integrating them into a transmedia application. We talk about why its more important to focus on value than on traffic, the laser and cover-your-bases platform strategies, and opine that Android, iOS, and the Web are the three viable platform choices. Giles finishes with a case-study of our work with a major airline.

Further Reading

Forging a New Alliance

with Scott Shaw

expand…

ThoughtWorks often organizes "Quarterly Technology Briefings" - open talks for the community in cities where we have offices. In this QTB in Toronto, Scott Shaw and I talk about how to build new relationships between IT and business. It explains why we think IT departments should be disbanded.

Technology in the Obama Campaign

with Zack Exley

expand…

My colleague Zack Exley and I talk about the software that was used by the 2008 Obama presidential campaign. An aspect I found particularly interesting was the way in which the software enabled and interacted with the organizational approach to the campaign.

Further Reading

What Does Tech Excellence Look Like?

To succeed with the modern digital business, you need a skillful technical organization. How does culture, talent, and technology mix to create that?

expand…

All the talk people make about transforming a business into a digital organization is all very well, but it can't happen unless you have a technical organization that can do its job well.

Nicole Forsgren has done research that correlates IT performance to Organizational Performance, and how her research on Devops identified three important indicators of IT performance: Deployment Frequency, Deployment Lead Time, and Mean Time To Recover. A simple example of lottery tickets illustrates the monetary value of rapid cycle times.

The characteristics we've observed about high-performing technical teams include: using Continuous Delivery, organized in a business oriented manner, technology led, and operating in a climate of trust. To go far you need to go in the right direction, but also take care of your vehicle.