Predicate Transfer: Efficient Pre-Filtering on Multi-Join Queries

TL;DR

This paper introduces predicate transfer, a method that enhances multi-join query performance by pre-filtering tables with Bloom filters, significantly reducing input sizes and outperforming Bloom join by 3.1x on TPC-H benchmarks.

Contribution

It generalizes Bloom join to multi-table joins using predicate transfer, inspired by Yannakakis' semi-join theory, enabling more efficient pre-filtering across complex join graphs.

Findings

Outperforms Bloom join by 3.1x on TPC-H benchmark

Effectively reduces join input sizes through pre-filtering

Generalizes semi-join principles to arbitrary join graphs

Abstract

This paper presents predicate transfer, a novel method that optimizes join performance by pre-filtering tables to reduce the join input sizes. Predicate transfer generalizes Bloom join, which conducts pre-filtering within a single join operation, to multi-table joins such that the filtering benefits can be significantly increased. Predicate transfer is inspired by the seminal theoretical results by Yannakakis, which uses semi-joins to pre-filter acyclic queries. Predicate transfer generalizes the theoretical results to any join graphs and use Bloom filters to replace semi-joins leading to significant speedup. Evaluation shows predicate transfer can outperform Bloom join by 3.1x on average on TPC-H benchmark.