Invertible Bloom Lookup Tables with Listing Guarantees

TL;DR

This paper introduces a worst-case analysis of Invertible Bloom Lookup Tables (IBLTs), providing constructions with guaranteed successful listing for sets up to a certain size, enhancing reliability in applications like network synchronization.

Contribution

It develops novel IBLT constructions with guaranteed listing success using coding theory, Steiner systems, and covering arrays, along with analyzing their size and memory requirements.

Findings

New IBLT constructions with guaranteed listing success.

Analysis of size and memory trade-offs for these IBLTs.

Verification of theoretical bounds through simulations.

Abstract

The Invertible Bloom Lookup Table (IBLT) is a probabilistic concise data structure for set representation that supports a listing operation as the recovery of the elements in the represented set. Its applications can be found in network synchronization and traffic monitoring as well as in error-correction codes. IBLT can list its elements with probability affected by the size of the allocated memory and the size of the represented set, such that it can fail with small probability even for relatively small sets. While previous works only studied the failure probability of IBLT, this work initiates the worst case analysis of IBLT that guarantees successful listing for all sets of a certain size. The worst case study is important since the failure of IBLT imposes high overhead. We describe a novel approach that guarantees successful listing when the set satisfies a tunable upper bound on its size. To allow that, we develop multiple constructions that are based on various coding techniques such as stopping sets and the stopping redundancy of error-correcting codes, Steiner systems, and covering arrays as well as new methodologies we develop. We analyze the sizes of IBLTs with listing guarantees obtained by the various methods as well as their mapping memory consumption. Lastly, we study lower bounds on the achievable sizes of IBLT with listing guarantees and verify the results in the paper by simulations.