Learned Query Superoptimization

TL;DR

This paper introduces learned query superoptimization, aiming to find optimal query plans through autonomous exploration, potentially improving performance for repetitive queries despite longer optimization times.

Contribution

It proposes a novel approach to query optimization using exploration-driven algorithms, Bayesian optimization, and program synthesis to discover optimal plans.

Findings

Potential to significantly accelerate repetitive query execution

Uses exploration-driven algorithms and Bayesian optimization

Enables autonomous discovery of optimal query plans

Abstract

Traditional query optimizers are designed to be fast and stateless: each query is quickly optimized using approximate statistics, sent off to the execution engine, and promptly forgotten. Recent work on learned query optimization have shown that it is possible for a query optimizer to "learn from its mistakes," correcting erroneous query plans the next time a plan is produced. But what if query optimizers could avoid mistakes entirely? This paper presents the idea of learned query superoptimization. A new generation of query superoptimizers could autonomously experiment to discover optimal plans using exploration-driven algorithms, iterative Bayesian optimization, and program synthesis. While such superoptimizers will take significantly longer to optimize a given query, superoptimizers have the potential to massively accelerate a large number of important repetitive queries being executed on data systems today.