Testing server calls in generated HTML
Wed 05 Jun 2024 15:00 CEST
Matteo Vaccari completes his article on testing template-generated HTML, by looking at how to use TDD with pages that make calls to the server.
Recent Changes
Here is a list of recent updates the site. You can also get this information as an RSS feed and I announce new articles on and Mastodon, X (Twitter), and LinkedIn.
I use this page to list both new articles and additions to existing articles. Since I often publish articles in installments, many entries on this page will be new installments to recently published articles, such announcements are indented and don’t show up in the recent changes sections of my home page.
Testing the behavior of generated HTML
Thu 30 May 2024 08:42
In the story so far, Matteo Vaccari has shown how to test the behaviour of the HTML templates, by checking the structure of the generated HTML. That's good, but what if we want to test the behavior of the HTML itself, plus any CSS and JavaScript it may use?
Parameterizing HTML template tests
Wed 29 May 2024 10:44
Testing templates for generating HTML leads to tests that are very similar. Matteo Vaccari wisely likes to separate the common elements of tests from those that vary. He continues his article to show how he does this by parameterizing the tests. The resulting tests are easier to write, and more importantly, faster to understand and modify.
Prefetching in Single-Page Applications
Wed 29 May 2024 10:24
Juntao Qiu's completes his set of data fetching patterns for single-page applications. Prefetching involves fetching data before it's called for in the application flow. Although this can mean data is fetched unnecessarily, it reduces latency should the data be needed.
Code Splitting in Single-Page Applications
Thu 23 May 2024 10:26
Single-Page Applications often require a lot of code to be downloaded to the browser, which can delay a page's initial appearance. Juntao Qiu's next pattern, Code Splitting, describes how this code can be divided up, so that modules are only loaded if they are going to be needed, and the dangers of doing so.
A short note on how I use and render footnotes
Wed 22 May 2024 14:17
Last week I added a small feature to this website, changing the way it renders footnotes. That prompted me to write this quick note about how I use footnotes, and how that influences the best way to render them.
Testing the contents of generated HTML
Wed 22 May 2024 10:14
Matteo Vaccari continues his testing of template-generated HTML by describing tests for the contents of that HTML. He shows how to gradually build up the template, using Test-Driven Development in Go and Java.
Using markup for fallbacks when fetching data
Tue 21 May 2024 11:30
Juntao Qiu's next data fetching pattern looks at how to specify fallback behavior using markup. This allows developers to pull such declarations out of the JavaScript components and into the markup they use while laying out the rest of the page. Juntao's React example shows how this works with the Suspense element, with a similar approach in vue.js.
Test-Driving HTML Templates
Tue 21 May 2024 11:06
When building a server-side rendered web application, it's valuable to test the HTML that's generated through templates. While these can be tested through end-to-end tests running in the browser, such tests are slow and more work to maintain than unit tests. My colleague Matteo Vaccari has written an article on how to use TDD to test drive these templates using xunit-style tools which can be run easily from the command line or as part of build scripts.
In this first installment Matteo describes how such tests can check the generated HTML for validity, with examples in Java and Go.
Parallel Data Fetching
Wed 15 May 2024 10:47
The second pattern in Juntao Qiu's series on data fetching is on how to avoid the dreaded Request Waterfall. Parallel Data Fetching queries multiple sources in parallel, so that the latency of the overall fetch is largest of the queries rather than the sum of them.
Data Fetching Patterns in Single-Page Applications
Tue 14 May 2024 09:49
Juntao Qiu is a thoughtful front-end developer experienced with the React programming environment. He's contributed a couple of useful articles to this site, describing helpful patterns for front-end programming. In this article he describes patterns for how single-page applications fetch data. This first installment describes how asynchronous queries can be wrapped in a handler to provide information about the state of the query.
photostream 131
Sun 12 May 2024 17:39
photostream 130
Sun 21 Apr 2024 18:21
Using data replication in legacy displacement
Wed 10 Apr 2024 11:00
Alessio Ferri and Tom Coggrave complete their article about introducing seams into mainframe systems by looking how we can use data replication. Done well, it can provide a rapid start to a displacement effort, but we must be wary of it coupling new systems to the legacy schema
Creating Seams in a Mainframe's Batch Pipelines
Thu 04 Apr 2024 09:39
Mainframe processing is often organized into pipelines of batch processes consuming and creating data files. Alessio Ferri and Tom Coggrave show a couple of ways they found to introduce seams into these pipelines, so that processing could be replaced by steps in a replacement platform.
Uncovering Seams in a Mainframe's external interfaces
Tue 02 Apr 2024 09:46
Alessio Ferri and Tom Coggrave move on to analyzing the internal seams of the application by treating the database as a coarse-grained seam. They describe how they introduced seams into the database readers and writers.
Joining LinkedIn
Thu 28 Mar 2024 12:26
As the enmuskification of Twitter continues, I’ve increasingly heard that more people are using LinkedIn to keep up with new professional material. So, a couple of weeks ago, I set up my LinkedIn account, so people can follow me on that platform.
I’ve always avoided LinkedIn - I’ve found the whole vibe of connections rather off-putting. I get too much spam from people wanting to connect as it is. But LinkedIn has added a “creator mode”, which encourages people to follow someone for posts rather than the bi-directional connection. It seems to be working reasonably well so far, so I’ve decided that I shall post all updates here to that account too. However I’m still avoiding connections, so please don’t send me a connection request unless we’ve done some substantial work together.
I’m still posting to X (Twitter), but as it steadily deteriorates, it gets less of my attention. If you follow me there, I recommend switching to one of my other feeds if you can.
I don’t particularly like LinkedIn for its feed, as I have little control over it. I never like the site suggestions, preferring as simple feeds of people. Sadly, LinkedIn doesn’t have lists, and pushes everything someone likes into the single connection feed. This leads to Too Many Posts which means I don’t pay attention to much.
Farewell, John Kordyback
Wed 27 Mar 2024 12:48
John Kordyback, a treasured colleague and friend, died last week, aged 64.
Uncovering Seams in a Mainframe's external interfaces
Wed 27 Mar 2024 10:36
Alessio Ferri and Tom Coggrave start detailing the seams they explored with two areas of external interfaces. Batch input of files are copied to new implementations while comparing the output of the processing pipelines. API access points can be covered with a proxy and traffic gradually directed to the new implementation.
Uncovering the seams in Mainframes for Incremental Modernisation
Tue 26 Mar 2024 09:32
Mainframe systems continue to run much of the world's computing workload, but it's often difficult to add new features to support growing business needs. Furthermore the architectural challenges that make them slow to enhance also make them hard to replace. To reduce the risk involved, we use an incremental approach to legacy displacement, gradually replacing legacy capabilities with implementations in modern technology. This strategy requires us to introduce seams into the mainframe system: points in which we could divert logic flow into newer services. In a recent project Alessio Ferri and Tom Coggrave investigated several approaches to introduce these seams into a long-lived mainframe system.
How to capture qualitative metrics
Tue 19 Mar 2024 09:33
Abi Noda and Tim Cochran complete their article on qualitative metrics by outlining how to capture them effectively. They discuss the mental steps that people go through as they respond to a survey and provide a template to get started when assessing developer experience. A final section looks at how qualitative and quantitative work together: often by starting with qualitative metrics to establish baselines and determine where to focus, followed with quantitative metrics to drill deeper into specific areas.
Code samples for the opening chapter of Refactoring
Fri 15 Mar 2024 11:04
From time to time people ask me for a copy of the code I used in the opening chapter of Refactoring, so they can follow along themselves. I had Reasons for not providing this code, specifically laziness. Fortunately Emily Bache is more dedicated, and she has set up a github repository - the Theatrical Players Refactoring Kata - with the code, and enough tests to make it reasonable to do the refactoring.
The repository goes further than this, however, in that it includes similar sample code in a dozen languages, including C, Java, Rust, and Python.
She has recently posted a video to her YouTube channel, which outlines why she encourages folks to use this code while they are reading that chapter. Her channel includes a lot of videos on good code technique, and she has a Patreon for readers to support her work.
The Benefits of Qualitative Metrics
Wed 13 Mar 2024 10:36
Abi Noda and Tim Cochran continue their discussion on using qualitative metrics to assess the productivity of development teams. In this installment they classify qualitative metrics into attitudinal and behavioral metrics. We also see that qualitative metrics allow you to measure things that are otherwise unmeasurable, provide missing visibility, and supply necessary context for quantitative data.
Measuring Developer Productivity via Humans
Tue 12 Mar 2024 09:36
Measuring developer productivity is a difficult challenge. Conventional metrics focused on development cycle time and throughput are limited, and there aren't obvious answers for where else to turn. Qualitative metrics offer a powerful way to measure and understand developer productivity using data derived from developers themselves. Abi Noda and Tim Cochran begin their discussion by explaining what a qualitative metric is and why we shouldn't reject them for being subjective or unreliable.
What if we rotate pairs every day?
Wed 06 Mar 2024 10:33
When pair programming, it's important to rotate the pairs frequently, but many organizations that do pair programming are reluctant to do that. Gabriel Robaina and Kieran Murphy ask the question: “What if we rotate pairs every day?” and worked with three teams through an exercise of daily pair rotation. They developed a lightweight methodology to help teams reflect on the benefits and challenges of pairing and how to solve them. Initial fears were overcome and teams discovered the benefits of frequently rotating pairs. They learned that pair swapping frequently greatly enhances the benefits of pairing. Their article shares the methodology they developed, their observations, and some common fears and insights shared by the participating team members.
Patterns of Legacy Displacement: Event Interception
Tue 05 Mar 2024 10:03
When we gradually replace a legacy system, we have plenty of cases where the legacy system and its replacement need to interact. Since these legacy systems are often difficult, and costly, to change, we need a mechanism that can integrate elements of the replacement while minimizing the impact to the legacy system. Ian Cartwright, Rob Horn, and James Lewis explain how we can use Event Interception on state-changing events, allowing us to forward them to the replacement.
Bliki: Periodic Face-to-Face
Tue 27 Feb 2024 09:17
Improvements in communications technology have led an increasing number of teams that work in a Remote-First style, a trend that was boosted by the forced isolation of Covid-19 pandemic. But a team that operates remotely still benefits from face-to-face gatherings, and should do them every few months.
Remote-first teams have everyone in a separate location, communicating entirely by email, chat, video and other communication tools. It has definite benefits: people can be recruited to the team from all over the world, and we can involve people with care-giving responsibilities. Wasteful hours of frustrating commutes can be turned into productive or recuperative time.
But however capable folks may be at remote working, and however nifty modern collaboration tools become, there is still nothing like being in the same place with the other members of a team. Human interactions are always richer when they are face-to-face. Video calls too easily become transactional, with little time for the chitchat that builds a proper human relationship. Without those deeper bonds, misunderstandings fester into serious relationship difficulties, and teams can get tangled in situations that would be effectively resolved if everyone were able to talk in person.
A regular pattern I see from those who are effective in remote-first work is that they ensure regular face-to-face meetings. During these they schedule those elements of work that are done better together. Remote work is more effective for tasks that require solo concentration, and modern tools can make remote pairing workable. But tasks that require lots of input from many people with rapid feedback are much easier to do when everyone is in the same room. No video-conference system can create the that depth of interaction, staring at a computer screen to see what other people are doing is draining, with no opportunity to pop out for a coffee together to break up the work. Debates about product strategy, explorations of systems architecture, explorations of new ground - these are common tasks for when the team is assembled.
For people to work effectively together they need to trust each other, aware of how much they can rely on each other. Trust is hard to develop online, where there isn't the social cues that can happen when we are in the same room. Thus the most valuable part of a face-to-face gathering isn't the scheduled work, it's chitchat while getting a coffee, and conviviality over lunch. Informal conversations, mostly not about work, forge the human contact that makes the work interactions be more effective.
Those guidelines suggest what the content for a face-to-face should be. Working together is both valuable in its own right, and an important part of team bonding. So we should set a full day of work, focusing on those tasks that benefit from the low-latency communication that comes from being together. We should then include what feels like too much time for breaks, informal chatter, and opportunities to step outside the office. I would avoid any artificial “team building” exercises, if only because of how much I hate them. Those who do gatherings like this stress the value from everyone energized afterwards, and thus able to be more effective in the following weeks.
Remote teams can be formed at large distances, and it's common to see members separated by hours of travel. For such teams, the rule of thumb I would use is to get together for a week every two or three months. After the team has become seasoned they may then decide to reduce the frequency, but I would worry if a team isn't having at least two face-to-face meetings a year. If a team is all in the same city, but using a remote-first style to reduce commuting, then they can organize shorter gatherings, and do them more frequently.
This kind of gathering may lead to rethinking of how to configure office space. Much has been made of how offices are far less used since the pandemic. Offices could well become less of a day-to-day workspace, and more a location for these kinds of irregular team gatherings. This leads to a need for flexible and comfortable team gathering spaces.
Some organizations may balk at the costs of travel and accommodation for a team assembly like this, but they should think of it as an investment in the team's effectiveness. Neglecting these face-to-faces leads to teams getting stuck, heading off in the wrong direction, plagued with conflict, and people losing motivation. Compared to this, saving on airplanes and hotels is a false economy.
Further Reading
Remote-first is one form of remote work, I explore the different styles of remote working and their trade-offs in Remote versus Co-located Work.
At Thoughtworks, we learned the importance of regular face-to-face gatherings for remote teams when we first started our offshore development centers nearly two decades ago. These generated the practices I describe in Using an Agile Software Process with Offshore Development.
Remote work, particularly when crossing time zones, puts a greater premium on asynchronous patterns of collaboration. My colleague Sumeet Moghe, a product manager, goes into depth on how to do this in his book The Async-First Playbook
Atlassian, a software product company, has recently entirely shifted to remote working, and published a report on its experiences. They have learned that it's wise for teams to have a face-to-face gathering roughly three times per year. Claire Lew surveyed remote-first teams in 2018, noting that a quarter of their respondents did retreats “several times a year”. 37Signals has operated as a remote-first company for nearly two decades and schedules meetups twice a year.
Acknowledgements
Alejandro Batanero, Andrew Thal, Chris Ford, Heiko Gerin, Kief Morris, Kuldeep Singh, Matt Newman, Michael Chaffee, Naval Prabhakar, Rafael Detoni, and Ramki Sitaraman discussed drafts of this post on our internal mailing list.
Engineering Practices for LLM Application Development
Tue 13 Feb 2024 12:22
LLM engineering involves much more than just prompt design or prompt engineering. Here David Tan and Jessie Wang reflect on how regular engineering practices such as testing and refactoring helped them deliver a prototype LLM application rapidly and reliably.
Onboarding bottleneck: final installment
Wed 31 Jan 2024 11:57
Tim and Prem finish their article on effective onboarding. They discuss the value of pair programming, setting up personal environments, and removing friction from the process.
Onboarding bottleneck: more steps for good onboarding
Tue 30 Jan 2024 09:33
Tim and Prem continue outlining the steps for an effective onboarding process. They talk about including new hires in the company culture, nailing the post-offer and first-day experience, and investing in self-service knowledge management