Efficient Data Management with a Flexible Address Space

TL;DR

This paper introduces a new storage engine enabling efficient in-place updates in sorted data files, reducing complexity and improving performance for data management applications.

Contribution

It presents a novel storage engine with a flexible address space that supports efficient in-place data modifications, simplifying data management.

Findings

Achieves high performance in key-value store benchmarks.

Reduces complexity compared to traditional indirection methods.

Supports arbitrary-sized data insertions and removals efficiently.

Abstract

Data management applications store their data using structured files in which data are usually sorted to serve indexing and queries. However, in-place insertions and removals of data are not naturally supported in a file's address space. To avoid repeatedly rewriting existing data in a sorted file to admit changes in place, applications usually employ extra layers of indirections, such as mapping tables and logs, to admit changes out of place. However, this approach leads to increased access cost and excessive complexity. This paper presents a novel storage engine that provides a flexible address space, where in-place updates of arbitrary-sized data, such as insertions and removals, can be performed efficiently. With this mechanism, applications can manage sorted data in a linear address space with minimal complexity. Extensive evaluations show that a key-value store built on top of it can achieve high performance and efficiency with a simple implementation.