A Model for Learned Bloom Filters, and Optimizing by Sandwiching

TL;DR

This paper models learned Bloom filters using machine learning, clarifies their guarantees, estimates optimal sizes, introduces a sandwiching optimization method, and extends the approach to Bloomier filters.

Contribution

It provides a formal model for learned Bloom filters, introduces a sandwiching optimization technique, and proposes a design approach for learned Bloomier filters.

Findings

Clarifies guarantees of learned Bloom filters.

Provides a method to estimate necessary learning function size.

Introduces sandwiching as an optimization technique.

Abstract

Recent work has suggested enhancing Bloom filters by using a pre-filter, based on applying machine learning to determine a function that models the data set the Bloom filter is meant to represent. Here we model such learned Bloom filters,, with the following outcomes: (1) we clarify what guarantees can and cannot be associated with such a structure; (2) we show how to estimate what size the learning function must obtain in order to obtain improved performance; (3) we provide a simple method, sandwiching, for optimizing learned Bloom filters; and (4) we propose a design and analysis approach for a learned Bloomier filter, based on our modeling approach.