SEER: Performance-Aware Leader Election in Single-Leader Consensus

TL;DR

SEER is a performance-aware leader election method for consensus systems that predicts and selects the optimal leader to minimize response times, improving efficiency without compromising safety.

Contribution

SEER introduces a dynamic, prediction-based leader election approach that adapts to system changes, enhancing performance in distributed consensus without altering core protocols.

Findings

Reduces mean response time by up to 32% compared to Raft.

Maintains safety properties of Raft despite dynamic leader re-elections.

Slight increase in leader reconfiguration frequency, tunable for performance-safety trade-offs.

Abstract

Modern stateful web services and distributed SDN controllers rely on log replication to omit data loss in case of fail-stop failures. In single-leader execution, the leader replica is responsible for ordering log updates and the initiation of distributed commits, in order to guarantee log consistency. Network congestions, resource-heavy computation, and imbalanced resource allocations may, however, result in inappropriate leader election and increased cluster response times. We present SEER, a logically centralized approach to performance prediction and efficient leader election in leader-based consensus systems. SEER autonomously identifies the replica that minimizes the average cluster response time, using prediction models trained dynamically at runtime. To balance the exploration and exploitation, SEER explores replicas' performance and updates their prediction models only after detecting significant system changes. We evaluate SEER in a traffic management scenario comprising [3..7] Raft replicas, and well-known data-center and WAN topologies. Compared to the Raft's uniform leader election, SEER decreases the mean control plane response time by up to ~32%. The benefit comes at the expense of the minimal adaptation of Raft election procedure and a slight increase in leader reconfiguration frequency, the latter being tunable with a guaranteed upper bound. No safety properties of Raft are invalidated by SEER.