Bonsai: Compiling Queries to Pruned Tree Traversals

TL;DR

This paper introduces a compiler that automatically generates optimized tree traversal code for complex queries, enabling faster data search and aggregation by generalizing pruning techniques.

Contribution

It automates the derivation of pruning conditions and fuses compound queries, supporting a wide range of predicates beyond standard ones, outperforming manual implementations.

Findings

Generated traversals match expert-written code.

Achieves asymptotic performance improvements over linear scans.

Supports a broad class of join predicates.

Abstract

Trees can accelerate queries that search or aggregate values over large collections. They achieve this by storing metadata that enables quick pruning (or inclusion) of subtrees when predicates on that metadata can prove that none (or all) of the data in a subtree affect the query result. Existing systems implement this pruning logic manually for each query predicate and data structure. We generalize and mechanize this class of optimization. Our method derives conditions for when subtrees can be pruned (or included wholesale), expressed in terms of the metadata available at each node. We efficiently generate these conditions using symbolic interval analysis, extended with new rules to handle geometric predicates (e.g., intersection, containment). Additionally, our compiler fuses compound queries (e.g., reductions on filters) into a single tree traversal. These techniques enable the automatic derivation of generalized single-index and dual-index tree joins that support a wide class of join predicates beyond standard equality and range predicates. The generated traversals match the behavior of expert-written code that implements query-specific traversals, and can asymptotically outperform the linear scans and nested-loop joins that existing systems fall back to when hand-written cases do not apply.