A Unified, Practical, and Understandable Model of Non-transactional Consistency Levels in Distributed Replication

TL;DR

This paper introduces a unified, intuitive model for non-transactional consistency levels in distributed systems, simplifying understanding and analysis of various protocols through a focus on ordering constraints.

Contribution

It proposes the SOP model that unifies consistency levels based on ordering constraints, and demonstrates its practical utility with a consistency checker for real systems.

Findings

The SOP model effectively captures common consistency levels.

The consistency checker analyzes real system histories.

The model clarifies the relationship between consistency levels and ordering constraints.

Abstract

We present a practical model of non-transactional consistency levels in the context of distributed data replication. Unlike prior work, our simple Shared Object Pool (SOP) model defines common consistency levels in a unified framework centered around the single concept of ordering. This naturally reflects modern cloud object storage services and is thus easy to understand. We show that a consistency level can be intuitively defined by specifying two types of constraints on the validity of orderings allowed by the level: convergence, which bounds the lineage shape of the ordering, and relationship, which bounds the relative positions between operations. We give examples of representative protocols and systems, and discuss their availability upper bound. To further demonstrate the expressiveness and practical relevance of our model, we use it to implement a Jepsen-integrated consistency checker for the four most common levels (linearizable, sequential, causal+, and eventual); the checker analyzes consistency conformity for small-scale histories of real system runs (etcd, ZooKeeper, and RabbitMQ).