How Good are Learned Cost Models, Really? Insights from Query Optimization Tasks

TL;DR

This paper systematically evaluates learned cost models in query optimization, revealing that traditional models often outperform LCMs in key tasks, and offers insights for future improvements.

Contribution

It provides a comprehensive comparison of LCMs and traditional models across core query optimization tasks, highlighting their relative performance and guiding future research.

Findings

Traditional cost models often outperform LCMs in query optimization tasks.

LCMs show promise but currently do not consistently surpass traditional models.

The study offers recommendations for enhancing LCM effectiveness in query optimization.

Abstract

Traditionally, query optimizers rely on cost models to choose the best execution plan from several candidates, making precise cost estimates critical for efficient query execution. In recent years, cost models based on machine learning have been proposed to overcome the weaknesses of traditional cost models. While these models have been shown to provide better prediction accuracy, only limited efforts have been made to investigate how well Learned Cost Models (LCMs) actually perform in query optimization and how they affect overall query performance. In this paper, we address this by a systematic study evaluating LCMs on three of the core query optimization tasks: join ordering, access path selection, and physical operator selection. In our study, we compare seven state-of-the-art LCMs to a traditional cost model and, surprisingly, find that the traditional model often still outperforms LCMs in these tasks. We conclude by highlighting major takeaways and recommendations to guide future research toward making LCMs more effective for query optimization.