State-Machine Replication Scalability Made Simple (Extended Version)

TL;DR

This paper introduces a generic method to transform leader-driven consensus protocols into scalable multi-leader protocols using a novel primitive, significantly improving throughput in large-scale distributed systems.

Contribution

The authors propose a new primitive called Sequenced Broadcast to enhance scalability of leader-driven protocols, enabling them to perform efficiently with many nodes.

Findings

Peak throughput of PBFT increased by 37x

HotStuff throughput increased by 56x

Raft throughput increased by 55x at 128 nodes

Abstract

Consensus, state-machine replication (SMR) and total order broadcast (TOB) protocols are notorious for being poorly scalable with the number of participating nodes. Despite the recent race to reduce overall message complexity of leader-driven SMR/TOB protocols, scalability remains poor and the throughput is typically inversely proportional to the number of nodes. We present Insanely Scalable State-Machine Replication, a generic construction to turn leader-driven protocols into scalable multi-leader ones. For our scalable SMR construction we use a novel primitive called Sequenced (Total Order) Broadcast (SB) which we wrap around PBFT, HotStuff and Raft leader-driven protocols to make them scale. Our construction is general enough to accommodate most leader-driven ordering protocols (BFT or CFT) and make them scale. Our implementation improves the peak throughput of PBFT, HotStuff, and Raft by 37x, 56x, and 55x, respectively, at a scale of 128 nodes.