Gen-DBA: Generative Database Agents

TL;DR

Gen-DBA introduces a versatile foundation model designed to optimize databases across diverse environments, overcoming the limitations of specialized ML models by offering broader generalization and adaptability.

Contribution

This paper proposes the concept and initial design of Gen-DBA, a unified foundation model for database optimization with agentic capabilities, addressing current ML4DB limitations.

Findings

Identifies limitations of narrow ML models in database optimization.

Proposes a vision and sketch design for a general-purpose foundation model.

Highlights research challenges for developing Gen-DBA.

Abstract

Leveraging Machine Learning to optimize database systems, referred to as Machine Learning for Databases (ML4DB, for short), dates back to the early 1990s, spanning indexing techniques, selectivity estimation, and query optimization. However, the idea has gained mainstream traction following the introduction of learned indexes in 2018, triggering a surge of research spanning learned indexes and cardinality estimators to learned query optimizers, storage layout design, resource management, and database tuning. The current ML4DB optimization landscape is dominated by narrow specialist ML models that are small and are trained on limited training data. Each specialist ML model targets a single database learning task on a fixed database engine, hardware platform, query workload, and optimization objective. As a result, they fall short in real-world settings, where these factors can vary significantly and evolve over time. This leads to an exponential number of ML models with limited portability and generalization capability, thus limiting the utility of existing ML4DB approaches. We address this limitation with Gen-DBA, a single general-purpose foundation model for optimizing databases with agentic capabilities. This paper presents the vision for Gen-DBA, provides a sketch design of how to realize it, and highlights several research challenges that need to be addressed to fully realize Gen-DBA.