Request Batch
Combine multiple requests to optimally utilise the network
Problem
If a lot of requests are sent to cluster nodes with a small amount of data, network latency and the request processing time (including serialization and deserialization of the request on the server side) can add significant overhead.
For example, in a network with 1gbps capacity, if the latency and request processing time is 100 microseconds and the client is sending hundreds of requests at the same time, it will significantly limit the overall throughput even though each request is just a few bytes.
Solution
Combine multiple requests together into a single request batch. The batch of the request will be sent to the cluster node for processing, with each request processed in exactly the same manner as an individual request. The node will then respond with a batch of responses.
for more details go to Chapter 31 of the online ebook at oreilly.com
This pattern is part of Patterns of Distributed Systems
23 November 2023