Nova-LSM: A Distributed, Component-based LSM-tree Key-value Store

TL;DR

Nova-LSM is a distributed, component-based LSM-tree key-value store that uses RDMA for communication, enabling elastic scalability and significantly outperforming traditional monolithic systems like LevelDB and RocksDB under skewed workloads.

Contribution

This paper introduces Nova-LSM, a novel disaggregated, component-based LSM-tree key-value store that leverages RDMA for communication and dynamic load balancing, achieving high scalability and performance.

Findings

Nova-LSM outperforms LevelDB and RocksDB by several orders of magnitude.

The system effectively balances load and parallelizes compaction dynamically.

Disaggregation enables elastic scalability and improved performance under skewed workloads.

Abstract

The cloud infrastructure motivates disaggregation of monolithic data stores into components that are assembled together based on an application's workload. This study investigates disaggregation of an LSM-tree key-value store into components that communicate using RDMA. These components separate storage from processing, enabling processing components to share storage bandwidth and space. The processing components scatter blocks of a file (SSTable) across an arbitrary number of storage components and balance load across them using power-of-d. They construct ranges dynamically at runtime to parallelize compaction and enhance performance. Each component has configuration knobs that control its scalability. The resulting component-based system, Nova-LSM, is elastic. It outperforms its monolithic counterparts, both LevelDB and RocksDB, by several orders of magnitude with workloads that exhibit a skewed pattern of access to data.