Scavenger+: Revisiting Space-Time Tradeoffs in Key-Value Separated LSM-trees

TL;DR

Scavenger+ enhances key-value separated LSM-trees by introducing efficient garbage collection, space-aware compaction, and adaptive scheduling, significantly improving performance and reducing space amplification in storage systems.

Contribution

It presents a novel framework, Scavenger+, that systematically reduces space amplification and improves performance in KV-separated LSM-trees through innovative strategies.

Findings

Reduces space amplification compared to existing methods.

Improves write performance significantly.

Efficiently adapts to system load for optimal resource use.

Abstract

Key-Value Stores (KVS) based on log-structured merge-trees (LSM-trees) are widely used in storage systems but face significant challenges, such as high write amplification caused by compaction. KV-separated LSM-trees address write amplification but introduce significant space amplification, a critical concern in cost-sensitive scenarios. Garbage collection (GC) can reduce space amplification, but existing strategies are often inefficient and fail to account for workload characteristics. Moreover, current key-value (KV) separated LSM-trees overlook the space amplification caused by the index LSM-tree. In this paper, we systematically analyze the sources of space amplification in KV-separated LSM-trees and propose Scavenger+, which achieves a better performance-space trade-off. Scavenger+ introduces (1) an I/O-efficient garbage collection scheme to reduce I/O overhead, (2) a space-aware compaction strategy based on compensated size to mitigate index-induced space amplification, and (3) a dynamic GC scheduler that adapts to system load to make better use of CPU and storage resources. Extensive experiments demonstrate that Scavenger+ significantly improves write performance and reduces space amplification compared to state-of-the-art KV-separated LSM-trees, including BlobDB, Titan, and TerarkDB.