Here is a list of recent updates the site. You can also get this information as an RSS feed and I announce new articles on twitter.
I use this page to list both new articles and additions to existing articles. Since I often publish articles in installments, many entries on this page will be new installments to recently published articles, such announcements are indented and don’t show up in the recent changes sections of my home page.
Thu 20 Jan 2022 10:50 EST
Yesterday Ian Cartwright, Rob Horn, and James Lewis described the Critical Aggregator and how it can metastasize into an invasive form. When a legacy system has such an Invasive Critical Aggregator it's often best, if a little counter-intuitive, to Divert the Flow of data by building a new critical aggregator first. Once this is done, we have far more freedom to change or relocate the various upstream data sources.
Wed 19 Jan 2022 10:53 EST
Business Leaders often need to make decisions that are influenced by a wide range of activity throughout the whole enterprise. To support this systems often have a what Ian Cartwright, Rob Horn, and James Lewis call a Critical Aggregator: a component whose job is to visit various other systems and pull this information together. A critical aggregator is important, but often metastasizes into an Invasive Critical Aggregator
Tue 18 Jan 2022 12:42 EST
Continuing his exploration of important patterns to maintain consistency across a cluster, Unmesh Joshi now looks at Two Phase Commit. It's broadly the most familiar approach, but comes with lots of complexities to make it work in practice over unreliable networks.
Wed 12 Jan 2022 10:11 EST
Ian Cartwright, Rob Horn, and James Lewis are also back with the New Year with a couple more articles from Patterns of Legacy Displacement in the funnel for the next couple of weeks. This one describes a Legacy Mimic: a part of the new system designed to make the old system think that nothing has changed.
Tue 11 Jan 2022 10:09 EST
One of the core challenges in a distributed system is keeping the state synchronized across all the nodes, especially when neither the nodes, or the connections between them, are reliable. The core approach to handle with is a replicated log: using the write-ahead log pattern over the cluster. Unmesh Joshi shows how this works using its most common implementation: the Raft protocol.
Wed 05 Jan 2022 10:53 EST
Unmesh Joshi is ready to start the New Year with a few more of his Patterns of Distributed Systems. With this one he attempts the tricky task of explaining Paxos. This is a well-known protocol developed by Leslie Lamport, for nodes to reach a reliable consensus when both they, and the network, are prone to unexpected failure. Although it's well-known, it's also notoriously difficult to understand, indeed we had considerable difficulty getting our heads around it. We hope this description makes it a bit easier for those who follow us.
Sun 26 Dec 2021 11:21 EST
I listen to a lot of interesting music, so picked out my six favorite new-to-me albums this year. I hope you find something interesting to groove to in there.
Wed 15 Dec 2021 10:10 EST
Andrew finishes his article on how to scale software architecture by looking at how this technique works in practice, and also outlines how things can go wrong.
Tue 14 Dec 2021 09:27 EST
Brandon finishes his article on how we should look at integration by arguing that it is a strategic element of an enterprise's infrastructure. You can buy the best products in the world but "none of it will make you competitive in a digital world if you continue to treat integration as a tactical nuisance to overcome so you take advantage of those new systems."
Mon 13 Dec 2021 10:05 EST
Andrew's fourth supporting element for the Advice Process is using a tech radar to capture and map out your local version of the technology trends you see across your organization.
Thu 09 Dec 2021 11:44 EST
Having architectural principles is not new, but in a world of highly-autonomous-teams they become essential because they are the means by which an aligned delivery direction is achieved without the need for control. In the latest installment, Andrew talks about what makes a good principle, and how they work with the Advice Process.
Thu 09 Dec 2021 10:35 EST
Thus far, Brandon has has explained why general purpose languages are better for integration. In this latest installment he explains that there are cases when commercial integration tools make sense
Wed 08 Dec 2021 10:07 EST
Andrew uses the Architecture Advisory Forum as a regular and recurring place and time for conversations. It's a weekly, hour-long, open invite meeting used to gather advice and share information across a broad group.
Tue 07 Dec 2021 09:50 EST
Many commercial integration tools market their ability to own the integration landscape and call out to general purpose languages as needed. While Brandon can appreciate the marketing behind such messaging — it promotes product penetration and lock-in — as architectural guidance, it is exactly backwards. Instead, we should almost always manage the interface evolution in a general purpose language for at least two reasons: so we can better manage the complexity of maintaining a clean interface, and so that we avoid the gravitational pull of our tool's mental model when making strategic integration decisions.
Wed 01 Dec 2021 10:59 EST
The Advice Process works when supported by four elements. Andrew describes the first of these, Decision Records, which act as a tool for thinking about and recording the decision process.
Wed 01 Dec 2021 10:50 EST
Brandon points out that while we have historically drawn up our project plans and costs around the boxes—the digital products we are introducing—the lines are the hidden and often primary driver of organizational tech debt. They are the reason that things just take longer now than they used to.
Tue 30 Nov 2021 09:45 EST
“Build versus buy” decisions are everywhere today, and rightly so. Building software is risky and expensive, and software product companies can spread that risk and expense across multiple customers. But my colleague Brandon Byars argues that the kinds of tools that are available to buy for systems integration are not products that directly solve a business problem.
Tue 30 Nov 2021 09:43 EST
Like many modern software architects, Andrew Harmel-Law struggles with the need to scale architectural thinking to larger organizations while allowing teams to be as autonomous as possible. The approach he's currently using is the "Advice Process", that encourages and supports these teams to be engaged in broader architectural decision making. In this first installment, Andrew describes this advice process, later installments will dig into four supporting elements that help make it work.
Wed 10 Nov 2021 13:10 EST
Evan Bottcher understands that good technical design decisions are very dependent on context. Teams that regularly work together on common goals are able to communicate regularly and negotiate changes quickly. These teams exhibit a strong force of alignment, and can make technology and design decisions that harness that strong force. As we zoom out in a larger organisation an increasingly weak force exists between teams and divisions that work independently and have less frequent collaboration. Recognising the differences in these strong and weak forces allows us to make better decisions and give better guidance for each level, allowing for more empowered teams that can move faster.
Wed 10 Nov 2021 13:04 EST
Within each normal-sized team, limit the choice of alternatives for any class of technology to three. These are: the current sensible default, the one we're experimenting with as a trial, and the one that we hate and want to retire.
The conversation goes like this: We want to introduce a new messaging technology. How many do we have already in place? Oh we have three in active use, including one that's considered legacy and we're partway through migrating off and one that we experimented with previously but didn't gain traction. Ok, so we're at our limit now. If we want to add another messaging tech then we have two choices. Either migrate all of our apps off the legacy tech, or properly rid ourselves of the failed experiment. This is quite closely related to the idea of capping the number of Innovation Tokens in use within your teams.
At a team level these kinds of limits are relatively easy to maintain and discuss and act upon, because we have common priorities and ways of working and high trust, high bandwidth communication. At the scope of the whole organisation the challenge is similar, but getting alignment takes a lot longer and doing actual migration and consolidation work can take a long time - so we sometimes have to allow for more variation in technology. We also use different techniques to discuss and communicate the status of our preferred technologies.
An approach we use at MYOB to engage our whole organisation in broader decisions about technology is by publishing our own MYOB Technology Radar, following the format of the Thoughtworks Technology Radar. This approach of building our own radar involves taking input from all of our verticals and teams, and making a clear statement on what technologies we encourage teams to adopt, trial, or more importantly which ones to keep clear of.
Tue 02 Nov 2021 09:39 EDT
Integrating the necessary security controls and audit capabilities to satisfy compliance requirements within a DevOps culture can capitalize on CI/CD pipeline automation, but presents unique challenges as an organization scales. Understanding the second order implications and unintended consequences caused by the chosen implementation is key to building an effective, secure, and scalable solution. My colleague Carl Nygard describes how to think of these choices through a series of four patterns for handling compliance.
Tue 26 Oct 2021 10:24 EDT
James Shore has revised his book "The Art of Agile Development". I'm pleased to write a foreword for this book as it is solid guide to learning how to get past faux-agile and develop the skills you need to get the benefits of the agile way of work.
Thu 14 Oct 2021 14:29 EDT
Bromley Mtn, VT (2021)
Wed 06 Oct 2021 10:30 EDT
Those of us developing software don’t need to be told what a big impact it’s had on humanity this century. I’ve long maintained that this places a serious responsibility on our profession. Whether asked to or not, we have a duty to ensure our systems don’t degrade our society. But in the tumult of so many software projects, it can be difficult to step back and understand the implications of our work. In the last few months my colleagues at Thoughtworks have been putting together a catalog of techniques that we can use to address this problem, which they published as the Responsible Tech Playbook.
The playbook is a free PDF download of about 50 slides, the bulk of which is a summary of a dozen tools and methods that teams can use to better understand their responsibilities. Each summary is a couple of slides outlining the basics of the technique: what is it, who created it, when we should use it, how it works, and our perspective on its place in our development efforts.
My colleagues highlighted three techniques that are particularly well-suited as first steps into this collection:
As these example suggest, these techniques are mostly ways to conduct structured workshops that help people explore different perspectives on their product. They also encourage engaging a wider range of stakeholders into the discussion and coming up with ways to monitor for bad outcomes and deal with them should they show up.
As professionals, we must take responsibility for the outcomes of our work - whether or not it matches our intention. We can’t avoid that responsibility by saying we are following our manager’s instructions, nor can we avoid the necessity of considering how our algorithms may lead to malignancies. Techniques like this help us explore our responsibilities and take thoughtful action to live up to them.
You can download the Responsible Tech Playbook as a PDF from the Thoughtworks web site.
Last year my colleague JoJo Swords wrote an article that explains more about why it matters that we understand our responsibility for our technology. In conjunction with that, Rebecca Parsons and Chad Wathington addressed the topic in a webinar.
Wed 08 Sep 2021 09:43 EDT
I've written a fair bit about how using pull requests can encourage a low integration frequency, increasing cycle time and discouraging refactoring. Rouan Wilsenach has had success using an approach that categorizes changes as Ship/Show/Ask - using this classification to decide whether and how to use pull requests.
Thu 26 Aug 2021 12:15 EDT
A couple of months ago I announced that I was stepping back from speaking. A few people wondered whether I would still be writing. I did indicate in that article that I am, but I felt it may be worth saying a bit more about what I’m concentrating on these days.
Tue 10 Aug 2021 10:51 EDT
We often need to access APIs from foreign codebases, and these foreign codebases usually have different vocabularies to ours. I've found it useful to encapsulate this interaction with a gateway that translates between our code and foreign code.
Thu 29 Jul 2021 09:49 EDT
To illustrate how these patterns work in practice, Ian, Rob, and James describe an example of how one of our teams used a number of Legacy Modernization Patterns to successfully replace integration middleware critical to the operation of their client's business as part of a larger legacy modernization programme. They combined patterns and refactorings to successfully manage risk to the business, and facilitate eating this particularly gristly part of the elephant.
Tue 27 Jul 2021 09:08 EDT
On many occasions when my colleagues find themselves talking to IT executives they hear how the executives have a suite of aging applications built using soon to be, if not already end of life technologies. More often that not these systems are hosted in costly data centers managed by 3rd parties and with inflexible contracts. These applications are critical to the successful operation of the business, while at the same time being one of the largest sources of business and operational risk.
One approach in this situation is to try to minimize the impact of replacement on the broader organization by 'simply' replacing the technology while leaving everything else 'as is'. Whilst Feature Parity often sounds like a reasonable proposition, we have learnt the hard way that people greatly underestimate the effort required, and thus misjudge the choice between this and the other alternatives.
Wed 21 Jul 2021 10:00 EDT
To do effective legacy displacement, we need to figure out how to break down the problem into manageable pieces. Extract Product Lines does this by identifying product lines and using them as the basis for migration.