Ark: A Real-World Consensus Implementation

TL;DR

Ark is a new consensus algorithm implementation that improves upon existing algorithms like Paxos and Raft, addressing data loss issues and supporting advanced features such as chained replication and unacknowledged writes.

Contribution

Ark introduces a novel consensus algorithm inspired by Raft, with enhancements for data safety and additional replication features, tailored for real-world database systems.

Findings

Fixes data loss issues in MongoDB's consensus algorithms

Supports chained replication and unacknowledged writes

Demonstrates improved reliability over existing algorithms

Abstract

Ark is an implementation of a consensus algorithm similar to Paxos and Raft, designed as an improvement over the existing consensus algorithm used by MongoDB and TokuMX. Ark was designed from first principles, improving on the election algorithm used by TokuMX, to fix deficiencies in MongoDB's consensus algorithms that can cause data loss. It ultimately has many similarities with Raft, but diverges in a few ways, mainly to support other features like chained replication and unacknowledged writes.