Time-Space Trade-Offs for Predecessor Search

TL;DR

This paper introduces a novel technique for establishing cell-probe lower bounds in static data structures, proving tight bounds for predecessor search and showing the optimality of van Emde Boas' data structure in certain space regimes.

Contribution

It presents the first explicit lower bounds breaking communication complexity barriers and establishes tight bounds for predecessor search, demonstrating the optimality of classic data structures in specific settings.

Findings

New lower bounds for predecessor search in static data structures.

Separation between polynomial and near linear space complexities.

Optimality of van Emde Boas' data structure for certain space and universe size conditions.

Abstract

We develop a new technique for proving cell-probe lower bounds for static data structures. Previous lower bounds used a reduction to communication games, which was known not to be tight by counting arguments. We give the first lower bound for an explicit problem which breaks this communication complexity barrier. In addition, our bounds give the first separation between polynomial and near linear space. Such a separation is inherently impossible by communication complexity. Using our lower bound technique and new upper bound constructions, we obtain tight bounds for searching predecessors among a static set of integers. Given a set Y of n integers of l bits each, the goal is to efficiently find predecessor(x) = max{y in Y | y <= x}, by representing Y on a RAM using space S. In external memory, it follows that the optimal strategy is to use either standard B-trees, or a RAM algorithm ignoring the larger block size. In the important case of l = c*lg n, for c>1 (i.e. polynomial universes), and near linear space (such as S = n*poly(lg n)), the optimal search time is Theta(lg l). Thus, our lower bound implies the surprising conclusion that van Emde Boas' classic data structure from [FOCS'75] is optimal in this case. Note that for space n^{1+eps}, a running time of O(lg l / lglg l) was given by Beame and Fich [STOC'99].