Recent Changes

Bliki: DiffDebugging

Mon 04 Dec 2023 00:00 EST

Regression bugs are newly appeared bugs in features of the software that have been around for a while. When hunting them, it usually valuable to figure out which change in the software caused them to appear. Looking at that change can give invaluable clues about where the bug is and how to squash it. There isn't a well-known term for this form of investigation, but I call it Diff Debugging.

Diff debugging only works if we have our code in version control, but fortunately these days that's the norm. But there are some more things that are needed to make it work effectively. We need Reproducible Builds, so that we can run old versions of the software easily. It helps greatly to have small commits, due to high-frequency integration. That way when we find the guilty commit, we can more easily narrow down what happened.

To find the commit that bred the bug, we begin by finding any past version without the bug. Mark this as a last-good version and the current version as the earliest-bad. Then find the commit half-way between the two and see if the bug is there. If so then this commit becomes the earliest-bad, otherwise it becomes the last-good. Repeat the process (which is a “half-interval” or “binary” search) until we've got the guilty commit.

If we use git, then the git bisect command will automate much of this for us. If we can write a test that will show the presence of the bug, then git bisect can use that too, automating the whole process of finding the guilty commit.

I often find diff debugging to be useful within a programming session. If I have slow tests that take a few minutes to run, I might program for half-an-hour running only a subset of the most relevant tests. As long as I commit after every green test run, I can use diff debugging should one of those slower tests fail. Such is the value of committing extremely frequently, even if they are so small that I feel its best to squash them for the long-term history. Some IDEs make this easier by keeping a local history automatically that is finer-grained than the commits to version control.

How to tackle unreliability of coding assistants

Tue 28 Nov 2023 10:21 EST

Over the last year, lots of developers have incorporated LLM coding assistants into their work, finding them a useful tool. But one of the problems of these tools is that they are unreliable, often coming up with poor or outright wrong-headed suggestions. Birgitta Böckeler continues her exploration of GenAI for developers by passing on what she's learned about how think about this unreliability, and why it may be good to call your LLM tool “Dusty”.

more…

Patterns of Distributed Systems is published by Pearson

Fri 24 Nov 2023 14:11 EST

During the last four years, my colleague Unmesh Joshi been developing a collection of patterns to help us all better understand how modern distributed systems work. We've been publishing drafts of these patterns on this site. Now these have turned into a book, published by Addison-Wesley in my signature series. As such, we've now removed the work-in-progress drafts from this site, and have replaced them with a catalog of pattern summaries. For those with a subscription to oreilly.com, we have deep links from the summaries to the relevant chapter of the online book.

more…

Three reasons a liberal arts degree helped me succeed in tech

Thu 09 Nov 2023 10:12 EST

My colleague Sannie Lee has met many students who are looking into getting into technology, taking narrow professionally-oriented majors. Sannie, however, has found that a traditional liberal-arts degree has given her skills that are highly relevant to her work as a product manager.

more…

Current thoughts on social media

Thu 02 Nov 2023 12:33 EDT

It's now been a year since The Muskover, what does my use of social media look like now, both as a reader and a writer?

more…

Headless Component: a pattern for composing React UIs

Wed 01 Nov 2023 10:34 EDT

As React UI controls become more sophisticated, complex logic can get intertwined with the visual representation. This makes it hard to reason about the behavior of the component, hard to test it, and necessary to build similar components that need a different look. Juntao Qiu tackles this by using a Headless Component, which extracts all non-visual logic and state management, separating the brain of a component from its looks.

more…

How is GenAI different from other code generators?

Tue 19 Sep 2023 08:57

How is code generation with GenAI different from more "traditional" code generators? The newest memo in Birgitta Böckeler's explorations of GenAI talks about abstraction levels in software engineering, and on which levels GenAI sits in the translation of our thoughts into zeros and ones.

more…

Demo Front-End: A front-end application to test and explore an API

Wed 23 Aug 2023 10:37 EDT

Many software teams create services exposed as APIs, designed to be consumed by other software and thus without any user-interface. Such services are hard to demonstrate, as they effectively just dump pages of JSON. A demo front-end is a simple user-interface just used to manipulate such an API. Matteo Vaccari describes how and why to build one - showing its usefulness both in explaining the API's capabilities to stakeholders and to help client developers explore how to interact with the API.

more…

Bottlenecks of Scaleups #05: Resilience and Observability

Tue 22 Aug 2023 10:21 EDT

Here is a new article in the bottlenecks of scaleups series, looking at resilience and observability. Startups tend to only address resilience when their systems are already down, often taking a very reactive approach. For a scaleup, excessive system downtime represents a significant bottleneck to the organization, both from the effort expended on restoring function and also from the impact of customer dissatisfaction. Punit Lad and Carl Nygard explain that to move past this, resilience needs to be built into the business objectives, which will influence the architecture, design, product management, and even governance of business systems.

more…

We will miss Stefan Tilkov

Mon 14 Aug 2023 14:58 EDT

We are saddened today to learn of the passing of Stefan Tilkov. We’ve met Stefan several times at workshops and conferences and always enjoyed his good company and insightful views. He advocated many of the things that we and our colleagues support - and as well as skillfully explaining these ideas, he also backed them up with concrete experience from his work. We shall really miss his wise contributions online, and regret that we won’t get the chance to chat with him again. We offer our condolences to his family, friends, and colleagues at INNOQ.

• Birgitta Boeckeler
• Erik Dörnenburg
• Dave Farley
• Neal Ford
• Martin Fowler
• James Lewis
• Sam Newman
• Rebecca Parsons
• Daniel Terhorst-North

Creating an integrated business and technology strategy

Tue 08 Aug 2023 16:05

My colleague Sarah Taraporewalla describes an approach to building technology strategy that challenges the convention. It starts by identifying the overall strategic directions that the organization is considering, and using common aspects of these directions to indicate the investigations needed for the organization to understand the technology implications raised by that strategic change. This first installment looks at two of these directions: expanding the business by creating complementary products, and expanding the business into new markets and regions.

more…

Bottlenecks of Scaleups #04: Cost Efficiency

Mon 31 Jul 2023 09:30 EDT

As startups begin to grow rapidly, they often find that early decisions that helped them find a product/market fit lead to excessive costs once traffic increases. These costs can threaten a scaleup's ability to grow. Sofia Tania and Stefania Stefansdottir have worked with many scaleups in this predicament and share their approach to understanding and reducing these costs.

more…

Exploring Gen AI - The toolchain

Wed 26 Jul 2023 10:52 EDT

My colleague Birgitta Böckeler has long been one of our senior technology leaders in Germany. She's now moved into a new role coordinating our work with Generative AI and its effect of software delivery practices. She's decided to publish her exploration in a series of memos. The first memo looks at the current toolchain for LLMs, categorizing them by what tasks they help with, how we interact with the LLM, and where they come from.

more…

Bliki: TeamTopologies

Tue 25 Jul 2023 09:25 EDT

Any large software effort, such as the software estate for a large company, requires a lot of people - and whenever you have a lot of people you have to figure out how to divide them into effective teams. Forming Business Capability Centric teams helps software efforts to be responsive to customers’ needs, but the range of skills required often overwhelms such teams. Team Topologies is a model for describing the organization of software development teams, developed by Matthew Skelton and Manuel Pais. It defines four forms of teams and three modes of team interactions. The model encourages healthy interactions that allow business-capability centric teams to flourish in their task of providing a steady flow of valuable software.

The primary kind of team in this framework is the stream-aligned team, a Business Capability Centric team that is responsible for software for a single business capability. These are long-running teams, thinking of their efforts as providing a software product to enhance the business capability.

Each stream-aligned team is full-stack and full-lifecycle: responsible for front-end, back-end, database, business analysis, feature prioritization, UX, testing, deployment, monitoring - the whole enchilada of software development. They are Outcome Oriented, focused on business outcomes rather than Activity Oriented teams focused on a function such as business analysis, testing, or databases. But they also shouldn't be too large, ideally each one is a Two Pizza Team. A large organization will have many such teams, and while they have different business capabilities to support, they have common needs such as data storage, network communications, and observability.

A small team like this calls for ways to reduce their cognitive load, so they can concentrate on supporting the business needs, not on (for example) data storage issues. An important part of doing this is to build on a platform that takes care of these non-focal concerns. For many teams a platform can be a widely available third party platform, such as Ruby on Rails for a database-backed web application. But for many products there is no single off-the-shelf platform to use, a team is going to have to find and integrate several platforms. In a larger organization they will have to access a range of internal services and follow corporate standards.

This problem can be addressed by building an internal platform for the organization. Such a platform can do that integration of third-party services, near-complete platforms, and internal services. Team Topologies classifies the team that builds this (unimaginatively-but-wisely) as a platform team.

Smaller organizations can work with a single platform team, which produces a thin layer over an externally provided set of products. Larger platforms, however, require more people than can be fed with two-pizzas. The authors are thus moving to describe a platform grouping of many platform teams.

An important characteristic of a platform is that it's designed to be used in a mostly self-service fashion. The stream-aligned teams are still responsible for the operation of their product, and direct their use of the platform without expecting an elaborate collaboration with the platform team. In the Team Topologies framework, this interaction mode is referred to as X-as-a-Service mode, with the platform acting as a service to the stream-aligned teams.

Platform teams, however, need to build their services as products themselves, with a deep understanding of their customer's needs. This often requires that they use a different interaction mode, one of collaboration mode, while they build that service. Collaboration mode is a more intensive partnership form of interaction, and should be seen as a temporary approach until the platform is mature enough to move to x-as-a service mode.

So far, the model doesn't represent anything particularly inventive. Breaking organizations down between business-aligned and technology support teams is an approach as old as enterprise software. In recent years, plenty of writers have expressed the importance of making these business capability teams be responsible for the full-stack and the full-lifecycle. For me, the bright insight of Team Topologies is focusing on the problem that having business-aligned teams that are full-stack and full-lifecycle means that they are often faced with an excessive cognitive load, which works against the desire for small, responsive teams. The key benefit of a platform is that it reduces this cognitive load.

This insight has profound implications. For a start it alters how platform teams should think about the platform. Reducing client teams' cognitive load leads to different design decisions and product roadmap to platforms intended primarily for standardization or cost-reduction. Beyond the platform this insight leads Team Topologies to develop their model further by identifying two more kinds of team.

Some capabilities require specialists who can put considerable time and energy into mastering a topic important to many stream-aligned teams. A security specialist may spend more time studying security issues and interacting with the broader security community than would be possible as a member of a stream-aligned team. Such people congregate in enabling teams, whose role is to grow relevant skills inside other teams so that those teams can remain independent and better own and evolve their services. To achieve this enabling teams primarily use the third and final interaction mode in Team Topologies. Facilitating mode involves a coaching role, where the enabling team isn't there to write and ensure conformance to standards, but instead to educate and coach their colleagues so that the stream-aligned teams become more autonomous.

Stream-aligned teams are responsible for the whole stream of value for their customers, but occasionally we find aspects of a stream-aligned team's work that is sufficiently demanding that it needs a dedicated group to focus on it, leading to the fourth and final type of team: complicated-subsystem team. The goal of a complicated-subsystem team is to reduce the cognitive load of the stream-aligned teams that use that complicated subsystem. That's a worthwhile division even if there's only one client team for that subsystem. Mostly complicated-subsystem teams strive to interact with their clients using x-as-a service mode, but will need to use collaboration mode for short periods.

Team Topologies includes a set of graphical symbols to illustrate teams and their relationships. These shown here are from the current standards, which differ from those used in the book. A recent article elaborates on how to use these diagrams.

Team Topologies is designed explicitly recognizing the influence of Conways Law. The team organization that it encourages takes into account the interplay between human and software organization. Advocates of Team Topologies intend its team structure to shape the future development of the software architecture into responsive and decoupled components aligned to business needs.

George Box neatly quipped: "all models are wrong, some are useful". Thus Team Topologies is wrong: complex organizations cannot be simply broken down into just four kinds of teams and three kinds of interactions. But constraints like this are what makes a model useful. Team Topologies is a tool that impels people to evolve their organization into a more effective way of operating, one that allows stream-aligned teams to maximize their flow by lightening their cognitive load.