Consistent Core
Maintain a smaller cluster providing stronger consistency to allow the large data cluster to coordinate server activities without implementing quorum-based algorithms.
Problem
When a cluster needs to handle a lot of data, it uses more and more servers. For a cluster of servers, there are some common requirements, such as selecting a server to be the master for a particular task, managing group membership information, mapping data partitions to servers etc. This functionality requires a strong consistency guarantee of linearizability. The implementation also needs to be fault-tolerant. A common approach is to use a fault-tolerant consensus algorithm based on Majority Quorum. However, in quorum-based systems, throughput degrades with the size of the cluster.
Solution
Implement a smaller cluster of 3 to 5 nodes which provides linearizability guarantee as well as fault tolerance. A separate data cluster can use the small consistent cluster to manage metadata and for taking cluster-wide decisions with primitives like Lease. This way, the data cluster can grow to a large number of servers but still be able to do certain actions that need strong consistency guarantees using the smaller metadata cluster.
for more details go to Chapter 25 of the online ebook at oreilly.com
This pattern is part of Patterns of Distributed Systems
23 November 2023