Characterize LSM-tree Compaction Performance via On-Device LLM Inference

TL;DR

This paper explores using on-device small-scale LLMs for real-time tuning of LSM-tree compaction parameters in key-value storage engines, balancing inference latency and reasoning accuracy to optimize performance.

Contribution

It evaluates the trade-off between model size, inference latency, and tuning effectiveness, and characterizes LSM-tree performance with parameter adjustments using RocksDB.

Findings

Positive correlation between model capability and tuning effectiveness

Small-scale LLMs can achieve effective real-time compaction tuning

Analysis of inference latency across different LLM scales

Abstract

Modern key-value storage engines built on Log-Structured Merge-trees (LSM-trees), such as RocksDB and LevelDB, rely heavily on the performance of their compaction operations, which are impacted by a complex set of interdependent configuration parameters. Manually tuning these parameters for optimal performance demands considerable expertise, while traditional auto-tuning approaches struggle with the enormous search space and low sample efficiency inherent to this domain. In recent years, Large Language Models (LLMs) have demonstrated strong capabilities in code generation and logical reasoning, offering new possibilities for system optimization. However, applying LLMs to real-time compaction tuning in such latency-sensitive environments is a double-edged sword. While large-scale LLMs can offer superior reasoning for strategy generation, their high inference latency and computational cost make them impractical for interactive, low-latency tuning. In contrast, small-scale LLMs achieve low latency but often at the expense of reasoning accuracy and tuning effectiveness. In this paper, we first evaluate this trade-off by analyzing the compaction-tuning performance and inference latency of LLMs at different scales in an LSM-tree-based tuning case. We then characterize the performance of LSM-tree on RocksDB v8.8.1, with a focus on adjusting the key compaction-related parameters under db_bench workloads. Our experimental results show a clear positive correlation between model capability and tuning effectiveness.