Compressing (Multidimensional) Learned Bloom Filters

TL;DR

This paper introduces a lossless input compression method for learned Bloom filters, significantly reducing memory usage while maintaining accuracy, especially effective with large datasets.

Contribution

It presents a novel lossless input compression technique that enhances memory efficiency of learned Bloom filters without sacrificing performance.

Findings

Significant memory reduction over existing learned Bloom filters

Effective compression especially with large datasets

Maintains comparable accuracy after compression

Abstract

Bloom filters are widely used data structures that compactly represent sets of elements. Querying a Bloom filter reveals if an element is not included in the underlying set or is included with a certain error rate. This membership testing can be modeled as a binary classification problem and solved through deep learning models, leading to what is called learned Bloom filters. We have identified that the benefits of learned Bloom filters are apparent only when considering a vast amount of data, and even then, there is a possibility to further reduce their memory consumption. For that reason, we introduce a lossless input compression technique that improves the memory consumption of the learned model while preserving a comparable model accuracy. We evaluate our approach and show significant memory consumption improvements over learned Bloom filters.