Hamava: Fault-tolerant Reconfigurable Geo-Replication on Heterogeneous Clusters

TL;DR

Hamava introduces a fault-tolerant, reconfigurable geo-replication protocol for heterogeneous clusters that supports dynamic membership and can utilize various local consensus mechanisms, enhancing scalability and throughput.

Contribution

The paper presents AVA, a novel reconfigurable geo-replication protocol that operates in heterogeneous environments with arbitrary failures and supports dynamic cluster membership.

Findings

Reconfiguration does not significantly impact transaction processing.

Heterogeneity can improve throughput.

Protocol is formally proven to be safe and live.

Abstract

Fault-tolerant replicated database systems consume less energy than the compute-intensive proof-of-work blockchain. Thus, they are promising technologies for the building blocks that assemble global financial infrastructure. To facilitate global scaling, clustered replication protocols are essential in orchestrating nodes into clusters based on proximity. However, the existing approaches often assume a homogeneous and fixed model in which the number of nodes across clusters is the same and fixed, and often limited to a fail-stop fault model. This paper presents heterogeneous and reconfigurable clustered replication for the general environment with arbitrary failures. In particular, we present AVA, a fault-tolerant reconfigurable geo-replication that allows dynamic membership: replicas are allowed to join and leave clusters. We formally state and prove the safety and liveness properties of the protocol. Furthermore, our replication protocol is consensus-agnostic, meaning each cluster can utilize any local replication mechanism. In our comprehensive evaluation, we instantiate our replication with both HotStuff and BFT-SMaRt. Experiments on geo-distributed deployments on Google Cloud demonstrates that members of clusters can be reconfigured without considerably affecting transaction processing, and that heterogeneity of clusters may significantly improve throughput.