Trie-Compressed Intersectable Sets

TL;DR

This paper presents space- and time-efficient algorithms for offline set intersection, achieving compressed storage and competitive performance compared to existing methods, with practical benefits demonstrated through experiments.

Contribution

It introduces novel algorithms and data structures for set intersection that are both space-efficient and fast, utilizing trie compression and the alternation measure.

Findings

Achieves compressed space representation of integer sets.

Supports k-way intersections in adaptive time proportional to the alternation measure.

Outperforms existing methods like Elias-Fano, Roaring Bitmaps, and RUP in experiments.

Abstract

We introduce space- and time-efficient algorithms and data structures for the offline set intersection problem. We show that a sorted integer set $S \subseteq [0{..}u)$ of $n$ elements can be represented using compressed space while supporting $k$-way intersections in adaptive $O(k\delta\lg{\!(u/\delta)})$ time, $\delta$ being the alternation measure introduced by Barbay and Kenyon. Our experimental results suggest that our approaches are competitive in practice, outperforming the most efficient alternatives (Partitioned Elias-Fano indexes, Roaring Bitmaps, and Recursive Universe Partitioning (RUP)) in several scenarios, offering in general relevant space-time trade-offs.