Request Pipeline
Improve latency by sending multiple requests on the connection without waiting for the response of the previous requests.
Problem
Communicating between servers within a cluster using Single-Socket Channel can cause performance issues if requests need to wait for responses to previous requests. To achieve better throughput and latency, the request queue on the server should be filled enough to make sure the server capacity is fully utilized. For example, when Singular Update Queue is used within a server, it can always accept more requests, until the queue fills up, while it's processing a request. If only one request is sent at a time, most of the server capacity is unnecessarily wasted.
Solution
Nodes send requests to other nodes without waiting for responses from previous requests. This is achieved by creating two separate threads, one for sending requests over a network channel and the other for receiving responses from the network channel.
for more details go to Chapter 32 of the online ebook at oreilly.com
This pattern is part of Patterns of Distributed Systems
23 November 2023