Characterizing and Adapting the Consistency-Latency Tradeoff in Distributed Key-value Stores

TL;DR

This paper presents probabilistic models and impossibility theorems for the CAP tradeoff, introduces PCAP systems that adapt to meet SLAs in real-time, and demonstrates their effectiveness in key-value stores like Cassandra and Riak.

Contribution

It introduces probabilistic CAP-like impossibility theorems and a novel adaptive system, PCAP, that optimizes consistency and latency in distributed key-value stores.

Findings

PCAP systems meet SLAs effectively in real-world deployments.

PCAP performs close to theoretical limits of the CAP tradeoff.

Extensions to geo-distributed data centers are successful.

Abstract

The CAP theorem is a fundamental result that applies to distributed storage systems. In this paper, we first present and prove two CAP-like impossibility theorems. To state these theorems, we present probabilistic models to characterize the three important elements of the CAP theorem: consistency (C), availability or latency (A), and partition tolerance (P). The theorems show the un-achievable envelope, i.e., which combinations of the parameters of the three models make them impossible to achieve together. Next, we present the design of a class of systems called PCAP that perform close to the envelope described by our theorems. In addition, these systems allow applications running on a single data-center to specify either a latency SLA or a consistency SLA. The PCAP systems automatically adapt, in real-time and under changing network conditions, to meet the SLA while optimizing the other C/A metric. We incorporate PCAP into two popular key-value stores -- Apache Cassandra and Riak. Our experiments with these two deployments, under realistic workloads, reveal that the PCAP system satisfactorily meets SLAs, and performs close to the achievable envelope. We also extend PCAP from a single data-center to multiple geo-distributed data-centers.