Optimizing Recursive Queries with Program Synthesis

TL;DR

This paper introduces a novel framework leveraging program synthesis tools to optimize recursive queries, significantly improving evaluation speed in data science and machine learning workloads.

Contribution

It presents the FGH-rule, a new optimization technique for recursive queries, utilizing program synthesizers and SMT-solvers for substantial performance gains.

Findings

Speedups up to 10,000x on Datalog systems

Effective optimization of recursive queries in data workloads

Demonstrated superiority over existing methods

Abstract

Most work on query optimization has concentrated on loop-free queries. However, data science and machine learning workloads today typically involve recursive or iterative computation. In this work, we propose a novel framework for optimizing recursive queries using methods from program synthesis. In particular, we introduce a simple yet powerful optimization rule called the "FGH-rule" which aims to find a faster way to evaluate a recursive program. The solution is found by making use of powerful tools, such as a program synthesizer, an SMT-solver, and an equality saturation system. We demonstrate the strength of the optimization by showing that the FGH-rule can lead to speedups up to 4 orders of magnitude on three, already optimized Datalog systems.