Finite Length Analysis on Listing Failure Probability of Invertible Bloom Lookup Tables

TL;DR

This paper analyzes the finite length behavior of Invertible Bloom Lookup Tables (IBLT), focusing on listing failure probability, and proposes a modification to hash functions to prevent small stopping sets, improving reliability.

Contribution

It introduces a stopping set analysis for IBLT, derives a recursive enumeration formula, and proposes SS avoiding hash functions to reduce failure probability.

Findings

Finite length analysis reveals error floor behavior.

Dominant stopping sets are of size 2 in the error floor region.

Proposed hash modification reduces listing failure probability.

Abstract

The Invertible Bloom Lookup Tables (IBLT) is a data structure which supports insertion, deletion, retrieval and listing operations of the key-value pair. The IBLT can be used to realize efficient set reconciliation for database synchronization. The most notable feature of the IBLT is the complete listing operation of the key-value pairs based on the algorithm similar to the peeling algorithm for low-density generator-matrix (LDGM) codes. In this paper, we will present a stopping set (SS) analysis for the IBLT which reveals finite length behaviors of the listing failure probability. The key of the analysis is enumeration of the number of stopping matrices of given size. We derived a novel recursive formula useful for computationally efficient enumeration. An upper bound on the listing failure probability based on the union bound accurately captures the error floor behaviors. It will be shown that, in the error floor region, the dominant SS have size 2. We propose a simple modification on hash functions, which are called SS avoiding hash functions, for preventing occurrences of the SS of size 2.