Linear Probing with Constant Independence

TL;DR

This paper investigates the performance of linear probing hash tables with different levels of hash function independence, showing that 5-wise independence guarantees constant expected operation time, resolving longstanding theoretical questions.

Contribution

It proves that 5-wise independent hash functions suffice for constant expected time in linear probing, improving understanding of hash function requirements.

Findings

Pairwise independence leads to logarithmic expected cost.

5-wise independence ensures constant expected cost.

Resolves open question on hash function independence for linear probing.

Abstract

Hashing with linear probing dates back to the 1950s, and is among the most studied algorithms. In recent years it has become one of the most important hash table organizations since it uses the cache of modern computers very well. Unfortunately, previous analysis rely either on complicated and space consuming hash functions, or on the unrealistic assumption of free access to a truly random hash function. Already Carter and Wegman, in their seminal paper on universal hashing, raised the question of extending their analysis to linear probing. However, we show in this paper that linear probing using a pairwise independent family may have expected {\em logarithmic} cost per operation. On the positive side, we show that 5-wise independence is enough to ensure constant expected time per operation. This resolves the question of finding a space and time efficient hash function that provably ensures good performance for linear probing.