Recent Changes
Here is a list of recent updates the site. You can also get this information as an RSS feed and I announce new articles on and Mastodon, X (Twitter), and LinkedIn.
I use this page to list both new articles and additions to existing articles. Since I often publish articles in installments, many entries on this page will be new installments to recently published articles, such announcements are indented and don’t show up in the recent changes sections of my home page.
Thu 05 Sep 2024 09:37 EDT
Decentralized data management requires automation to scale governance
effectively. Fitness functions are a powerful automated governance
technique my colleagues have applied to data products within the context
of a Data Mesh. Since data products serve as the foundational building
blocks of a data strategy, ensuring robust governance around them
significantly increases the chances of success. Kiran
Prakash explains how to do this, starting with simple tests for
key architectural characteristics and moving on to leveraging metadata
and Large Language Models.
more…
Wed 04 Sep 2024 00:00 EDT
Cycle Time is a measure of how long it takes to get a new feature in a
software system from idea to running in production. In Agile circles, we try
to minimize cycle time. We do this by defining and implementing very small
features and minimizing delays in the development process. Although the rough
notion of cycle time, and the importance of reducing it, is common, there is a
lot of variations on how cycle time is measured.
A key characteristic of agile software development is a shift from a
Waterfall Process, where work is decomposed based on
activity (analysis, coding, testing) to an Iterative Process where work is
based on a subset of functionality (simple pricing, bulk discount,
valued-customer discount). Doing this generates a feedback loop where we can learn
from putting small features in front of users. This learning allows us to
improve our development process and allows us to better understand where the
software product can provide value for our customers.
This feedback is a core benefit of an iterative approach, and like most
such feedback loops, the quicker I get the feedback, the happier I am. Thus
agile folks put a lot of emphasis on how fast we can get a feature through the
entire workflow and into production. The phrase cycle time is a measure of that.
But here we run into difficulties. When do we start and stop the clock on
cycle time?
The stopping time is the easiest, most glibly it's when the feature is put
into production and helping its users. But there are circumstances where this
can get muddy. If a team is using a Canary Release, should it
be when used by the first cohort, or only when released to the full
population? Do we count only when the app store has approved its release, thus
adding an unpredictable delay that's mostly outside the control of the
development team?.
The start time has even more variations. A common marker is when a
developer makes a first commit to that feature, but that ignores any time
spent in preparatory analysis. Many people would go further back and say:
“when the customer first has the idea for a feature”. This is all very well
for a high priority feature, but how about something that isn't that urgent,
and thus sits in a triage area for a few weeks before being ready to enter
development. Do we start the clock when the team first places the feature on
the card wall
and we start to seriously work on it?
I also run into the phase lead time, sometimes instead of
“cycle time”, but often together - where people make a distinction between the
two, often based on a different start time. However there isn't any
consistency between how people distinguish between them. So in general, I
treat “lead time” as a synonym to “cycle time”, and if someone is using both,
I make sure I understand how that individual is making the distinction.
The different bands of cycle time all have their advantages, and it's often
handy to use different bands in the same situation, to highlight differences.
In that situation, I'd use a distinguishing adjective (e.g. “first-commit cycle
time” vs “idea cycle time”) to tell them apart. There's no generally accepted
terms for such adjectives, but I think they are better than trying to
create a distinction between “cycle time” and “lead time”.
What these questions tell us is that cycle time, while a useful concept, is
inherently slippery. We should be wary of comparing cycle times between teams,
unless we can be confident we have consistent notions of their stop and start times.
But despite this, thinking in terms of cycle time, and trying to minimize
it, is a useful activity. It's usually worthwhile to build a value stream map
that shows every step from idea to production, identifying the steps in the
work flow, how much time is spent on them, and how much waiting between them.
Understanding this flow of work allows us to find ways to reduce the cycle
time. Two commonly effective interventions are to reduce the size of features
and (counter-intuitively) increase Slack. Doing the work to
understand flow to improve it is worthwhile because
the faster we get ideas into production, the more
rapidly we gain the benefits of the new features, and get the feedback to
learn and improve our ways of working.
Acknowledgements
Andrew Harmel-Law, Chris Ford, James Lewis, José Pinar, Kief Morris, Manoj Kumar M, Matteo
Vaccari, and Rafael Ferreira discussed this post
on our internal mailing list
Thu 22 Aug 2024 11:51 EDT
Two decades ago, I posted that I found that the strangler fig plant was an interesting metaphor for the gradual replacement of a legacy system. I didn’t refer to the metaphor since, but meanwhile it grew a life of its own. Other people increasingly referred to the strangler fig approach to modernization, and traffic to that post steadily increased: currently it gets about 5000 page views a month, one of the more popular pages on this site. So I decided I needed to update that page, and have rewritten it, focusing on the core activities we need to do to make a success of such a venture.
(This summarizes more detailed writing from Ian Cartwright, Rob Horn, and James Lewis.)
Thu 15 Aug 2024 10:32 EDT
Much of the attention to generative AI in software development is
about generating code. But it may have a more useful role in helping us
understand existing code. This is especially true for older codebases
that are getting hard to maintain (“legacy”) or to improve onboarding in
teams that have a lot of fluctuation.
Birgitta Böckeler demonstrates the possibilities here
by picking an issue from an open-source hospital management system and
exploring how AI could help her deal with it.
more…
Wed 31 Jul 2024 16:11 EDT
From time to time I take a look at my site analytics to see how much traffic various bits of this site get. When doing this I saw that I continue to get a lot of traffic to the Catalog of Patterns of Enterprise Application Architecture. I put this together not long after writing the book, and it’s rather minimal. Since it still gets traffic I felt it was time for some sprucing up. The content is the same, summaries of the patterns in the book. I’ve changed the main catalog page to include the pattern intents. I’ve also added deep links to the book on oreilly.com, so if you have a subscription to that, it will jump directly to that text. Hopefully these changes will make the catalog page a bit more useful.
Tue 30 Jul 2024 12:10 EDT
There's a lot of discussion about using regulation to restrict the use
of AI and other software algorithms. I think that the better regulation
would be to ensure that decisions made by software must be explainable.
more…
Wed 05 Jun 2024 15:00
Matteo Vaccari completes his article on testing
template-generated HTML, by looking at how to use TDD with pages that make
calls to the server.
more…
Thu 30 May 2024 08:42 EDT
In the story so far, Matteo Vaccari has shown how
to test the behaviour of the HTML templates,
by checking the structure of the generated HTML. That's good, but what
if we want to test the behavior of the HTML itself, plus any CSS and
JavaScript it may use?
more…
Wed 29 May 2024 10:44 EDT
Testing templates for generating HTML leads to tests that are very
similar. Matteo Vaccari wisely likes to separate the
common elements of tests from those that vary. He continues
his article to show how he does this by
parameterizing the tests. The resulting tests are
easier to write, and more importantly, faster to understand and modify.
more…
Wed 29 May 2024 10:24 EDT
Juntao Qiu's completes his set of data fetching
patterns for single-page applications. Prefetching involves fetching data
before it's called for in the application flow. Although this can mean
data is fetched unnecessarily, it reduces latency should the data be needed.
more…
Thu 23 May 2024 10:26 EDT
Single-Page Applications often require a lot of code to be downloaded
to the browser, which can delay a page's initial
appearance. Juntao Qiu's next pattern, Code Splitting,
describes how this code can be divided up, so that modules are only
loaded if they are going to be needed, and the dangers of doing so.
more…
Wed 22 May 2024 14:17 EDT
Last week I added a small feature to this website, changing the way it
renders footnotes. That prompted me to write this quick note about how I
use footnotes, and how that influences the best way to render them.
more…
Wed 22 May 2024 10:14 EDT
Matteo Vaccari continues his testing of
template-generated HTML by describing tests for the contents of that
HTML. He shows how to gradually build up the template, using Test-Driven
Development in Go and Java.
more…
Tue 21 May 2024 11:30 EDT
Juntao Qiu's next data fetching pattern looks at how to specify
fallback behavior using markup. This allows developers to
pull such declarations out of the JavaScript components and into the
markup they use while laying out the rest of the page. Juntao's React
example shows how this works with the Suspense element, with a similar
approach in vue.js.
more…
Tue 21 May 2024 11:06 EDT
When building a server-side rendered web application, it's valuable
to test the HTML that's generated through templates. While these can be
tested through end-to-end tests running in the browser, such tests are
slow and more work to maintain than unit tests. My colleague
Matteo Vaccari has written an article on how to use TDD
to test drive these templates using xunit-style tools which can be run
easily from the command line or as part of build scripts.
In this first installment Matteo describes how such tests can check the
generated HTML for validity, with examples in Java and Go.
more…
Wed 15 May 2024 10:47 EDT
The second pattern in Juntao Qiu's series on data
fetching is on how to avoid the dreaded Request Waterfall. Parallel Data
Fetching queries multiple sources in parallel, so that the latency of the
overall fetch is largest of the queries rather than the sum of them.
more…
Tue 14 May 2024 09:49 EDT
Juntao Qiu is a thoughtful front-end
developer experienced with the React programming environment. He's
contributed a couple of useful articles to this site, describing helpful
patterns for front-end programming. In this article he describes patterns
for how single-page applications fetch data. This first installment
describes how asynchronous queries can be wrapped in a handler to provide
information about the state of the query.
more…
Sun 12 May 2024 17:39 EDT
Sun 21 Apr 2024 18:21 EDT
Wed 10 Apr 2024 11:00 EDT
Alessio Ferri and Tom Coggrave
complete their article about introducing seams into mainframe systems by
looking how we can use data replication. Done well, it can provide a
rapid start to a displacement effort, but we must be wary of it
coupling new systems to the legacy schema
more…
Thu 04 Apr 2024 09:39 EDT
Mainframe processing is often organized into pipelines of batch
processes consuming and creating data files. Alessio
Ferri and Tom Coggrave show a couple of ways
they found to introduce seams into these pipelines, so that processing
could be replaced by steps in a replacement platform.
more…
Tue 02 Apr 2024 09:46 EDT
Alessio Ferri and Tom Coggrave move
on to analyzing the internal seams of the application by treating
the database as a coarse-grained seam. They describe how they introduced
seams into the database readers and writers.
more…
Thu 28 Mar 2024 12:26 EDT
As the enmuskification of Twitter continues, I’ve increasingly heard that more people are using LinkedIn to keep up with new professional material. So, a couple of weeks ago, I set up my LinkedIn account, so people can follow me on that platform.
I’ve always avoided LinkedIn - I’ve found the whole vibe of connections rather off-putting. I get too much spam from people wanting to connect as it is. But LinkedIn has added a “creator mode”, which encourages people to follow someone for posts rather than the bi-directional connection. It seems to be working reasonably well so far, so I’ve decided that I shall post all updates here to that account too. However I’m still avoiding connections, so please don’t send me a connection request unless we’ve done some substantial work together.
I’m still posting to X (Twitter), but as it steadily deteriorates, it gets less of my attention. If you follow me there, I recommend switching to one of my other feeds if you can.
I don’t particularly like LinkedIn for its feed, as I have little control over it. I never like the site suggestions, preferring as simple feeds of people. Sadly, LinkedIn doesn’t have lists, and pushes everything someone likes into the single connection feed. This leads to Too Many Posts which means I don’t pay attention to much.
Wed 27 Mar 2024 12:48 EDT
John Kordyback, a treasured colleague and friend, died last week, aged 64.
more…
Wed 27 Mar 2024 10:36 EDT
Alessio Ferri and Tom Coggrave start
detailing the seams they explored with two areas of external interfaces.
Batch input of files are copied to new implementations while comparing the output of the
processing pipelines. API access points can be covered with a proxy and
traffic gradually directed to the new implementation.
more…
Tue 26 Mar 2024 09:32 EDT
Mainframe systems continue to run much of the world's computing workload,
but it's often difficult to add new features to support growing business
needs. Furthermore the architectural challenges that make them slow to
enhance also make them hard to replace. To reduce the risk involved, we
use an incremental approach to legacy displacement, gradually replacing
legacy capabilities with implementations in modern technology. This
strategy requires us to introduce seams into the mainframe system: points
in which we could divert logic flow into newer services. In a recent
project Alessio Ferri and Tom Coggrave
investigated several approaches to introduce these seams into a
long-lived mainframe system.
more…
Tue 19 Mar 2024 09:33 EDT
Abi Noda and Tim Cochran complete
their article on qualitative metrics by outlining how to capture them
effectively. They discuss the mental steps that people go through as
they respond to a survey and provide a template to get started when assessing
developer experience. A final section looks at how qualitative and
quantitative work together: often by starting with qualitative metrics to
establish baselines and determine where to focus, followed with
quantitative metrics to drill deeper into specific areas.
more…
Fri 15 Mar 2024 11:04 EDT
From time to time people ask me for a copy of the code I used in the
opening chapter of Refactoring, so
they can follow along themselves. I had Reasons for not providing this code,
specifically laziness. Fortunately Emily Bache is more dedicated, and she
has set up a github repository - the Theatrical
Players Refactoring Kata - with the code, and enough tests to make it
reasonable to do the refactoring.
The repository goes further than this, however, in that it includes similar
sample code in a dozen languages, including C, Java, Rust, and Python.
She has recently posted a video to her YouTube
channel, which outlines why she encourages folks to use this code while
they are reading that chapter. Her channel includes a lot of videos on good
code technique, and she has a Patreon for readers to support
her work.
Wed 13 Mar 2024 10:36 EDT
Abi Noda and Tim Cochran continue their
discussion on using qualitative metrics to assess the productivity of
development teams. In this installment they classify qualitative metrics into
attitudinal and behavioral metrics. We also see that qualitative metrics
allow you to measure things that are otherwise unmeasurable, provide missing
visibility, and supply necessary context for quantitative data.
more…
Tue 12 Mar 2024 09:36 EDT
Measuring developer productivity is a difficult challenge.
Conventional metrics focused on development cycle time and throughput are
limited, and there aren't obvious answers for where else to turn.
Qualitative metrics offer a powerful way to measure and understand
developer productivity using data derived from developers themselves.
Abi Noda and Tim Cochran begin their
discussion by explaining what a qualitative metric is and
why we shouldn't reject them for being subjective or unreliable.
more…