A Compiler for Operations on Relations with Bag Semantics

TL;DR

This paper introduces an abstract loop-based IR for relational algebra on sets and bags, enabling fusion and optimization, leading to significant speedups and high-quality code generation for database operations.

Contribution

It presents a novel IR that abstracts loops from data structures, supporting complex operators and enabling fusion for efficient code generation.

Findings

Achieved up to 3.87x speedup on benchmarks

Generated code with performance comparable to Hyper on TPC-H

Demonstrated portability across different data structures

Abstract

We describe an abstract loop-based intermediate representation that can express fused implementations of relational algebra expressions on sets and bags (multisets). The loops are abstracted away from physical data structures thus making it easier to generate, reason about, and perform optimization like fusion on. The IR supports the natural relational algebra as well as complex operators that are used in production database systems, including outer joins, non-equi joins, and differences. We then show how to compile this IR to efficient C++ code that co-iterates over the physical data structures present in the relational algebra expression. Our approach lets us express fusion across disparate operators, leading to a 3.87x speedup (0.77--12.23x) on selected LSQB benchmarks and worst-case optimal triangle queries. We also demonstrate that our compiler generates code of high quality: it has similar sequential performance to Hyper on TPC-H with a 1.00x speedup (0.38--4.34x) and competitive parallel performance with a 0.61x speedup (0.23--1.80x). Finally, our approach is portable across data structures.