Accelerating LSM-Tree with the Dentry Management of File System

TL;DR

This paper introduces DeLSM, a novel LSM-tree design that replaces SST files with directories of individual KV files, significantly reducing disk I/O during compactions and improving performance.

Contribution

DeLSM reorganizes SST files into directories of KV files, transforming compaction into directory entry transfers, which reduces disk I/O and enhances LSM-tree efficiency.

Findings

DeLSM outperforms existing LSM-tree variants in various benchmarks.

Significant reduction in disk I/O during compactions.

Improved overall performance of key-value stores.

Abstract

The log-structured merge tree (LSM-tree) gains wide popularity in building key-value (KV) stores. It employs logs to back up arriving KV pairs and maintains a few on-disk levels with exponentially increasing capacity limits, resembling a tiered tree-like structure. A level comprises SST files, each of which holds a sequence of sorted KV pairs. From time to time, LSM-tree redeploys KV pairs from a full level to the lower level by compaction, which merge-sorts and moves KV pairs among SST files, thereby incurring substantial disk I/Os. In this paper, we revisit the design of LSM-tree and find that organizing multiple KV pairs in an SST file entails the heavyweight redeployment of actual KV pairs in a compaction. Accordingly we revolutionize the organization of KV pairs by transforming an SST file of KV pairs to an SST directory, in which each KV pair makes into an independent KV file with the key and value as filename and main file contents, respectively. Moving KV pairs in a compaction converts to transferring directory entries (dentrys), which causes concretely fewer disk I/Os. This is the essence of our design named DeLSM. We build a prototype of DeLSM on LevelDB and evaluation results show that it significantly outperforms the state-of-the-art LSM-tree variants in different dimensions.