Scaling Out Acid Applications with Operation Partitioning

TL;DR

This paper introduces Operation Partitioning, a novel approach that enables scalable, ACID-compliant OLTP applications by partitioning operations rather than data, using static analysis and a lock-free protocol, demonstrated with significant performance improvements.

Contribution

It proposes Operation Partitioning, a new paradigm for scaling OLTP applications that maintains ACID guarantees through operation-based partitioning and a lock-free coordination protocol.

Findings

Up to 4.2x increase in throughput

Latency reduced by up to 58.6x

Stronger serializability guarantees

Abstract

OLTP applications with high workloads that cannot be served by a single server need to scale out to multiple servers. Typically, scaling out entails assigning a different partition of the application state to each server. But data partitioning is at odds with preserving the strong consistency guarantees of ACID transactions, a fundamental building block of many OLTP applications. The more we scale out and spread data across multiple servers, the more frequent distributed transactions accessing data at different servers will be. With a large number of servers, the high cost of distributed transactions makes scaling out ineffective or even detrimental. In this paper we propose Operation Partitioning, a novel paradigm to scale out OLTP applications that require ACID guarantees. Operation Partitioning indirectly partitions data across servers by partitioning the application's operations through static analysis. This partitioning of operations yields to a lock-free Conveyor Belt protocol for distributed coordination, which can scale out unmodified applications running on top of unmodified database management systems. We implement the protocol in a system called Elia and use it to scale out two applications, TPC-W and RUBiS. Our experiments show that Elia can increase maximum throughput by up to 4.2x and reduce latency by up to 58.6x compared to MySQL Cluster while at the same time providing a stronger isolation guarantee (serializability instead of read committed).