Towards a General Framework for ML-based Self-tuning Databases

TL;DR

This paper explores applying ML-based self-tuning methods to FoundationDB, highlighting challenges like invalid configurations, and finds that simple random search performs competitively compared to complex ML techniques.

Contribution

It identifies key challenges in applying ML to database tuning and provides empirical comparison showing random search's effectiveness alongside Bayesian optimization and reinforcement learning.

Findings

ML methods improved FoundationDB throughput by up to 38%

Random search was only 4% worse than complex ML methods

Challenges like unknown parameter ranges are crucial for ML adoption in databases

Abstract

Machine learning (ML) methods have recently emerged as an effective way to perform automated parameter tuning of databases. State-of-the-art approaches include Bayesian optimization (BO) and reinforcement learning (RL). In this work, we describe our experience when applying these methods to a database not yet studied in this context: FoundationDB. Firstly, we describe the challenges we faced, such as unknown valid ranges of configuration parameters and combinations of parameter values that result in invalid runs, and how we mitigated them. While these issues are typically overlooked, we argue that they are a crucial barrier to the adoption of ML self-tuning techniques in databases, and thus deserve more attention from the research community. Secondly, we present experimental results obtained when tuning FoundationDB using ML methods. Unlike prior work in this domain, we also compare with the simplest of baselines: random search. Our results show that, while BO and RL methods can improve the throughput of FoundationDB by up to 38%, random search is a highly competitive baseline, finding a configuration that is only 4% worse than the, vastly more complex, ML methods. We conclude that future work in this area may want to focus more on randomized, model-free optimization algorithms.