Learning to Optimize Join Queries With Deep Reinforcement Learning

TL;DR

This paper introduces a deep reinforcement learning approach to optimize join query planning, outperforming traditional heuristics and existing optimizers in various database systems.

Contribution

It presents a novel RL-based join optimizer that learns to generate efficient query plans, adaptable to specific datasets and workloads, with implementations in multiple DBMSs.

Findings

DQ achieves comparable plan quality to native optimizers.

DQ can execute queries orders of magnitude faster after learning.

The approach integrates easily into existing database systems.

Abstract

Exhaustive enumeration of all possible join orders is often avoided, and most optimizers leverage heuristics to prune the search space. The design and implementation of heuristics are well-understood when the cost model is roughly linear, and we find that these heuristics can be significantly suboptimal when there are non-linearities in cost. Ideally, instead of a fixed heuristic, we would want a strategy to guide the search space in a more data-driven way---tailoring the search to a specific dataset and query workload. Recognizing the link between classical Dynamic Programming enumeration methods and recent results in Reinforcement Learning (RL), we propose a new method for learning optimized join search strategies. We present our RL-based DQ optimizer, which currently optimizes select-project-join blocks. We implement three versions of DQ to illustrate the ease of integration into existing DBMSes: (1) A version built on top of Apache Calcite, (2) a version integrated into PostgreSQL, and (3) a version integrated into SparkSQL. Our extensive evaluation shows that DQ achieves plans with optimization costs and query execution times competitive with the native query optimizer in each system, but can execute significantly faster after learning (often by orders of magnitude).