[Extended Version] ArceKV: Towards Workload-driven LSM-compactions for Key-Value Store Under Dynamic Workloads

TL;DR

This paper introduces ArceKV, a workload-driven key-value store that uses flexible compaction strategies to adapt to dynamic workloads, significantly improving performance over existing methods.

Contribution

It presents ElasticLSM for flexible LSM-tree management and Arce, a lightweight engine for optimal compaction decisions, enabling better adaptation to workload changes.

Findings

ArceKV achieves approximately 3x faster performance in dynamic workloads.

The proposed approach outperforms state-of-the-art strategies across diverse scenarios.

Flexible compaction management improves overall system adaptability and efficiency.

Abstract

Key-value stores underpin a wide range of applications due to their simplicity and efficiency. Log-Structured Merge Trees (LSM-trees) dominate as their underlying structure, excelling at handling rapidly growing data. Recent research has focused on optimizing LSM-tree performance under static workloads with fixed read-write ratios. However, real-world workloads are highly dynamic, and existing workload-aware approaches often struggle to sustain optimal performance or incur substantial transition overhead when workload patterns shift. To address this, we propose ElasticLSM, which removes traditional LSM-tree structural constraints to allow more flexible management actions (i.e., compactions and write stalls) creating greater opportunities for continuous performance optimization. We further design Arce, a lightweight compaction decision engine that guides ElasticLSM in selecting the optimal action from its expanded action space. Building on these components, we implement ArceKV, a full-fledged key-value store atop RocksDB. Extensive evaluations demonstrate that ArceKV outperforms state-of-the-art compaction strategies across diverse workloads, delivering around 3x faster performance in dynamic scenarios.