Aleph Filter: To Infinity in Constant Time

TL;DR

Aleph Filter is a new data structure that maintains constant-time operations and optimal memory-FPR trade-offs for dynamic set-membership queries, regardless of data growth.

Contribution

It introduces Aleph Filter, supporting all operations in constant time and providing memory-FPR trade-offs comparable to static filters for growing data.

Findings

Supports insertions, queries, deletes in constant time

Maintains stable performance and false positive rate

Offers optimal memory-FPR trade-offs for dynamic data

Abstract

Filter data structures are widely used in various areas of computer science to answer approximate set-membership queries. In many applications, the data grows dynamically, requiring their filters to expand along with the data. However, existing methods for expanding filters cannot maintain stable performance, memory footprint, and false positive rate (FPR) simultaneously. We address this problem with Aleph Filter, which makes the following contributions. (1) It supports all operations (insertions, queries, deletes, etc.) in constant time, no matter how much the data grows. (2) Given an estimate of how much the data will ultimately grow, Aleph Filter provides a memory vs. FPR trade-offs on par with static filters.