MultiPaxos Made Complete

TL;DR

This paper provides a comprehensive, detailed implementation and analysis of MultiPaxos, addressing previous gaps in guidance, and demonstrating improved performance, stability, and robustness in practical distributed systems.

Contribution

It offers the first complete, step-by-step design, pseudocode, and open-source implementation of MultiPaxos, enhancing its practical utility and correctness.

Findings

Improved performance stability and resource efficiency.

Enhanced leader election and failure detection mechanisms.

Better liveness under network partitions.

Abstract

MultiPaxos, while a fundamental Replicated State Machine algorithm, suffers from a dearth of comprehensive guidelines for achieving a complete and correct implementation. This deficiency has hindered MultiPaxos' practical utility and adoption and has resulted in flawed claims about its capabilities. Our paper aims to bridge the gap between MultiPaxos' complexity and practical implementation through a meticulous and detailed design process spanning more than a year. It carefully dissects each phase of MultiPaxos and offers detailed step-by-step pseudocode -- in addition to a complete open-source implementation -- for all components, including the leader election, the failure detector, and the commit phase. The implementation of our complete design also provides better performance stability, resource usage, and network partition tolerance than naive MultiPaxos versions. Our specification includes a lightweight log compaction approach that avoids taking repeated snapshots, significantly improving resource usage and performance stability. Our failure detector, integrated into the commit phase of the algorithm, uses variable and adaptive heartbeat intervals to settle on a better leader under partial connectivity and network partitions, improving liveness under such conditions.