Hash Adaptive Bloom Filter

TL;DR

The paper introduces Hash Adaptive Bloom Filter (HABF), a novel probabilistic data structure that customizes hash functions for improved accuracy and efficiency in membership testing, addressing limitations of traditional Bloom filters.

Contribution

HABF allows hash function customization for positive keys, enhancing performance and flexibility over standard Bloom filters, with theoretical analysis and extensive experimental validation.

Findings

HABF reduces false positive rates compared to standard Bloom filters.

HABF improves construction and query times.

HABF uses less memory while maintaining accuracy.

Abstract

Bloom filter is a compact memory-efficient probabilistic data structure supporting membership testing, i.e., to check whether an element is in a given set. However, as Bloom filter maps each element with uniformly random hash functions, few flexibilities are provided even if the information of negative keys (elements are not in the set) are available. The problem gets worse when the misidentification of negative keys brings different costs. To address the above problems, we propose a new Hash Adaptive Bloom Filter (HABF) that supports the customization of hash functions for keys. The key idea of HABF is to customize the hash functions for positive keys (elements are in the set) to avoid negative keys with high cost, and pack customized hash functions into a lightweight data structure named HashExpressor. Then, given an element at query time, HABF follows a two-round pattern to check whether the element is in the set. Further, we theoretically analyze the performance of HABF and bound the expected false positive rate. We conduct extensive experiments on representative datasets, and the results show that HABF outperforms the standard Bloom filter and its cutting-edge variants on the whole in terms of accuracy, construction time, query time, and memory space consumption (Note that source codes are available in [1]).