Evolving NoSQL Databases Without Downtime

TL;DR

This paper introduces KVolve, an extension for Redis that enables online, lazy data format evolution supporting high-availability without significant downtime or performance overhead.

Contribution

KVolve provides a practical, expressive solution for online data evolution in NoSQL databases, supporting complex updates with minimal disruption.

Findings

Supports substantial data format changes in Redis

Imposes minimal overhead during normal operation

Enables online updates without long pauses

Abstract

NoSQL databases like Redis, Cassandra, and MongoDB are increasingly popular because they are flexible, lightweight, and easy to work with. Applications that use these databases will evolve over time, sometimes necessitating (or preferring) a change to the format or organization of the data. The problem we address in this paper is: How can we support the evolution of high-availability applications and their NoSQL data online, without excessive delays or interruptions, even in the presence of backward-incompatible data format changes? We present KVolve, an extension to the popular Redis NoSQL database, as a solution to this problem. KVolve permits a developer to submit an upgrade specification that defines how to transform existing data to the newest version. This transformation is applied lazily as applications interact with the database, thus avoiding long pause times. We demonstrate that KVolve is expressive enough to support substantial practical updates, including format changes to RedisFS, a Redis-backed file system, while imposing essentially no overhead in general use and minimal pause times during updates.