Automatic Database Configuration Debugging using Retrieval-Augmented Language Models

TL;DR

Andromeda leverages retrieval-augmented language models to automate database configuration debugging, providing accurate diagnostic suggestions by integrating domain-specific context from various sources, thus outperforming existing methods.

Contribution

The paper introduces Andromeda, a novel framework using retrieval-augmented language models for automatic DBMS configuration debugging, incorporating domain-specific context retrieval to enhance diagnostic accuracy.

Findings

Andromeda significantly outperforms existing solutions in real-world datasets.

The retrieval-augmented strategy improves the relevance of diagnostic suggestions.

Effective document retrieval and telemetry analysis are key to the framework's success.

Abstract

Database management system (DBMS) configuration debugging, e.g., diagnosing poorly configured DBMS knobs and generating troubleshooting recommendations, is crucial in optimizing DBMS performance. However, the configuration debugging process is tedious and, sometimes challenging, even for seasoned database administrators (DBAs) with sufficient experience in DBMS configurations and good understandings of the DBMS internals (e.g., MySQL or Oracle). To address this difficulty, we propose Andromeda, a framework that utilizes large language models (LLMs) to enable automatic DBMS configuration debugging. Andromeda serves as a natural surrogate of DBAs to answer a wide range of natural language (NL) questions on DBMS configuration issues, and to generate diagnostic suggestions to fix these issues. Nevertheless, directly prompting LLMs with these professional questions may result in overly generic and often unsatisfying answers. To this end, we propose a retrieval-augmented generation (RAG) strategy that effectively provides matched domain-specific contexts for the question from multiple sources. They come from related historical questions, troubleshooting manuals and DBMS telemetries, which significantly improve the performance of configuration debugging. To support the RAG strategy, we develop a document retrieval mechanism addressing heterogeneous documents and design an effective method for telemetry analysis. Extensive experiments on real-world DBMS configuration debugging datasets show that Andromeda significantly outperforms existing solutions.