KV-Tandem -- a Modular Approach to Building High-Speed LSM Storage Engines

TL;DR

KV-Tandem introduces a modular architecture for LSM storage engines that enhances functionality and performance by leveraging simple KVSs, with practical implementation showing 3-4x speed improvements over RocksDB.

Contribution

The paper proposes KV-Tandem, a novel modular architecture with LSM bypass for building high-speed, feature-rich storage engines on simple KVSs, and demonstrates its effectiveness in XDP-Rocks.

Findings

XDP-Rocks achieves 3x to 4x performance improvements over RocksDB.

KV-Tandem enables advanced features like range queries and snapshots.

XDP-Rocks is deployed in production, reducing operator costs.

Abstract

We present~\emph{KV-Tandem}, a modular architecture for building LSM-based storage engines on top of simple, non-ordered persistent key-value stores (KVSs). KV-Tandem enables advanced functionalities such as range queries and snapshot reads, while maintaining the native KVS performance for random reads and writes. Its modular design offers better performance trade-offs compared to previous KV-separation solutions, which struggle to decompose the monolithic LSM structure. Central to KV-Tandem is~\emph{LSM bypass} -- a novel algorithm that offers a fast path to basic operations while ensuring the correctness of advanced APIs. We implement KV-Tandem in \emph{XDP-Rocks}, a RocksDB-compatible storage engine that leverages the XDP KVS and incorporates practical design optimizations for real-world deployment. Through extensive microbenchmark and system-level comparisons, we demonstrate that XDP-Rocks achieves 3x to 4x performance improvements over RocksDB across various workloads. XDP-Rocks is already deployed in production, delivering significant operator cost savings consistent with these performance gains.