Gradient-Based Join Ordering

TL;DR

This paper introduces a novel gradient-based method for join ordering in databases, leveraging differentiable cost models and continuous plan relaxation to improve efficiency and effectiveness.

Contribution

It proposes a differentiable approach to join ordering using continuous relaxations and Graph Neural Networks, enabling gradient-based optimization of query plans.

Findings

Gradient-based approach finds lower-cost plans than traditional methods.

Runtime scales better than discrete search algorithms.

Demonstrates effectiveness on two graph datasets.

Abstract

Join ordering is the NP-hard problem of selecting the most efficient order in which to evaluate joins (conjunctive, binary operators) in a database query. Because query execution performance critically depends on this choice, join ordering lies at the core of query optimization. Traditional approaches cast this problem as a discrete combinatorial search over binary trees guided by a cost model, but they have trade-offs between effectiveness and efficiency. We show that when the cost model is differentiable, query plans can be continuously relaxed into a soft adjacency matrix that represents a superposition of plans. This continuous relaxation, combined with differentiable constraints that enforce plan validity, enables a gradient-based search for low-cost plans within this relaxed space. Using a Graph Neural Network as the cost model, we demonstrate that this gradient-based approach can find comparable and even lower-cost plans compared to traditional discrete search methods on two different graph datasets. Furthermore, we empirically show that the runtime of this approach scales better than discrete search algorithms. We believe this first step towards gradient-based join ordering can lead to more effective and efficient query optimizers in the future.