Parachute: Single-Pass Bi-Directional Information Passing

TL;DR

Parachute introduces a method for enabling single-pass bi-directional information passing in database query plans, improving execution times by analyzing data flow and using fingerprint columns, thus bridging the gap between uni-directional passing and optimal algorithms.

Contribution

It presents a novel approach to achieve bi-directional information passing in a single pass by static analysis and fingerprint columns, enhancing query performance.

Findings

Improves execution time by 1.54x on JOB benchmark.

Achieves 1.24x speedup with semi-join filtering.

Utilizes 15% extra space for performance gains.

Abstract

Sideways information passing is a well-known technique for mitigating the impact of large build sides in a database query plan. As currently implemented in production systems, sideways information passing enables only a uni-directional information flow, as opposed to instance-optimal algorithms, such as Yannakakis'. On the other hand, the latter require an additional pass over the input, which hinders adoption in production systems. In this paper, we make a step towards enabling single-pass bi-directional information passing during query execution. We achieve this by statically analyzing between which tables the information flow is blocked and by leveraging precomputed join-induced fingerprint columns on FK-tables. On the JOB benchmark, Parachute improves DuckDB v1.2's end-to-end execution time without and with semi-join filtering by 1.54x and 1.24x, respectively, when allowed to use 15% extra space.