Facilitating Database Tuning with Hyper-Parameter Optimization: A Comprehensive Experimental Evaluation

TL;DR

This paper provides a comprehensive evaluation of hyper-parameter optimization techniques for database tuning, demonstrating their effectiveness and identifying optimal algorithms for different tuning modules, along with a cost-efficient benchmarking approach.

Contribution

It offers a broad evaluation of configuration tuning algorithms, integrating hyper-parameter optimization methods into database tuning, and introduces a unified benchmarking framework.

Findings

Hyper-parameter optimization algorithms improve database tuning performance.

Different algorithms are optimal for different tuning modules.

A surrogate-based benchmark reduces evaluation costs significantly.

Abstract

Recently, using automatic configuration tuning to improve the performance of modern database management systems (DBMSs) has attracted increasing interest from the database community. This is embodied with a number of systems featuring advanced tuning capabilities being developed. However, it remains a challenge to select the best solution for database configuration tuning, considering the large body of algorithm choices. In addition, beyond the applications on database systems, we could find more potential algorithms designed for configuration tuning. To this end, this paper provides a comprehensive evaluation of configuration tuning techniques from a broader perspective, hoping to better benefit the database community. In particular, we summarize three key modules of database configuration tuning systems and conduct extensive ablation studies using various challenging cases. Our evaluation demonstrates that the hyper-parameter optimization algorithms can be borrowed to further enhance the database configuration tuning. Moreover, we identify the best algorithm choices for different modules. Beyond the comprehensive evaluations, we offer an efficient and unified database configuration tuning benchmark via surrogates that reduces the evaluation cost to a minimum, allowing for extensive runs and analysis of new techniques.