LatentTune: Efficient Tuning of High Dimensional Database Parameters via Latent Representation Learning

TL;DR

LatentTune is a novel machine learning-based method that efficiently tunes high-dimensional database parameters by learning a latent representation, significantly improving performance and reducing tuning time across multiple database workloads.

Contribution

It introduces a latent space learning approach with data augmentation and external metrics integration to optimize full parameter configurations efficiently.

Findings

Achieves up to 1332% performance improvement on RocksDB.

Realizes 11.82% throughput gain and 46.01% latency reduction on MySQL.

Outperforms baseline models across four different workloads.

Abstract

As data volumes continue to grow, optimizing database performance has become increasingly critical, making the implementation of effective tuning methods essential. Among various approaches, database parameter tuning has proven to be a highly effective means of enhancing performance. Recent studies have shown that machine learning techniques can successfully optimize database parameters, leading to significant performance improvements. However, existing methods still face several limitations. First, they require substantial time to generate large training datasets. Second, to cope with the challenges of highdimensional optimization, they typically optimize only a subset of parameters rather than the full configuration space. Third, they often rely on information from similar workloads instead of directly leveraging information from the target workload. To address these limitations, we propose LatentTune, a novel approach that differs fundamentally from traditional methods. To reduce the time required for data generation, LatentTune incorporates a data augmentation strategy. Furthermore, it constructs a latent space that compresses information from all database parameters, enabling the optimization of the full configuration space. In addition, LatentTune integrates external metric information into the latent space, allowing for precise tuning tailored to the actual target workload. Experimental results demonstrate that LatentTune outperforms baseline models across four workloads on MySQL and RocksDB, achieving up to 1332% improvement for RocksDB and 11.82% throughput gain with 46.01% latency reduction for MySQL.