Automated Training of Learned Database Components with Generative AI

TL;DR

This paper investigates using generative AI models like GPT to synthesize training data for learned database components, aiming to improve database optimization techniques by augmenting datasets with realistic query and plan data.

Contribution

It presents the first feasibility study on employing generative models to create training data for database components, addressing data scarcity challenges.

Findings

Generative models can produce realistic query distributions.

Synthetic data improves adaptability of learned database techniques.

Initial results show promising augmentation benefits.

Abstract

The use of deep learning for database optimization has gained significant traction, offering improvements in indexing, cardinality estimation, and query optimization. However, acquiring high-quality training data remains a significant challenge. This paper explores the possibility of using generative models, such as GPT, to synthesize training data for learned database components. We present an initial feasibility study investigating their ability to produce realistic query distributions and execution plans for database workloads. Additionally, we discuss key challenges, such as data scalability and labeling, along with potential solutions. The initial results suggest that generative models can effectively augment training datasets, improving the adaptability of learned database techniques.