Optimize Cardinality Estimation Model Pretraining by Simplifying the Training Datasets

TL;DR

This paper improves cardinality estimation model pretraining by identifying the most impactful datasets through Group DRO, and demonstrates that a significantly simplified dataset can achieve comparable performance in zero-shot scenarios.

Contribution

It introduces a method to select impactful training datasets using Group DRO and proposes a simplified dataset for pretraining that maintains performance.

Findings

Simplified dataset achieves similar zero-shot performance as larger datasets.

Group DRO helps identify datasets that contribute most to model performance.

Pretraining with reduced datasets is effective for cardinality estimation.

Abstract

The cardinality estimation is a key aspect of query optimization research, and its performance has significantly improved with the integration of machine learning. To overcome the "cold start" problem or the lack of model transferability in learned cardinality estimators, some pre-training cardinality estimation models have been proposed that use learning across multiple datasets and corresponding workloads. These models typically train on a dataset created by uniformly sampling from many datasets, but this approach may not be optimal. By applying the Group Distributionally Robust Optimization (Group DRO) algorithm to training datasets, we find that some specific training datasets contribute more significantly to model performance than others. Based on this observation, we conduct extensive experiments to delve deeper into pre-training cardinality estimators. Our results show how the performance of these models can be influenced by the datasets and corresponding workloads. Finally, we introduce a simplified training dataset, which has been reduced to a fraction of the size of existing pretraining datasets. Sufficient experimental results demonstrate that the pre-trained cardinality estimator based on this simplified dataset can still achieve comparable performance to existing models in zero-shot setups.