Rethinking LSM-tree based Key-Value Stores: A Survey

TL;DR

This survey reviews recent advancements in LSM-tree based key-value stores, focusing on optimization techniques addressing performance variability, space efficiency, and multi-tenant architectures, and discusses future research directions.

Contribution

It provides a comprehensive review of recent LSM-tree optimization methods, highlighting challenges and opportunities in modern distributed key-value store systems.

Findings

Summarizes recent optimization techniques for LSM-trees

Identifies challenges in performance, space, and multi-tenancy

Suggests future research directions in LSM-tree systems

Abstract

LSM-tree is a widely adopted data structure in modern key-value store systems that optimizes write performance in write-heavy applications by using append writes to achieve sequential writes. However, the unpredictability of LSM-tree compaction introduces significant challenges, including performance variability during peak workloads and in resource-constrained environments, write amplification caused by data rewriting during compactions, read amplification from multi-level queries, trade-off between read and write performance, as well as efficient space utilization to mitigate space amplification. Prior studies on LSM-tree optimizations have addressed the above challenges; however, in recent years, research on LSM-tree optimization has continued to propose. The goal of this survey is to review LSM-tree optimization, focusing on representative works in the past five years. This survey first studies existing solutions on how to mitigate the performance impact of LSM-tree flush and compaction and how to improve basic key-value operations. In addition, distributed key-value stores serve multi-tenants, ranging from tens of thousands to millions of users with diverse requirements. We then analyze the new challenges and opportunities in these modern architectures and across various application scenarios. Unlike the existing survey papers, this survey provides a detailed discussion of the state-of-the-art work on LSM-tree optimizations and gives future research directions.