Optimal Succinct Rank Data Structure via Approximate Nonnegative Tensor Decomposition

TL;DR

This paper introduces a new succinct rank data structure with improved space-time tradeoffs, leveraging approximate nonnegative tensor decomposition to achieve near-optimal redundancy and query efficiency.

Contribution

It presents a novel connection between succinct data structures and approximate nonnegative tensor decomposition, enabling improved space and query time tradeoffs.

Findings

Achieves space redundancy of n/(\log n)^{ ext{Omega}(t)} plus lower order terms.

Matches the cell-probe lower bound for space-time tradeoffs.

Provides explicit tensor approximation leading to explicit data structure construction.

Abstract

Given an $n$-bit array $A$, the succinct rank data structure problem asks to construct a data structure using space $n+r$ bits for $r\ll n$, supporting rank queries of form $\mathtt{rank}(x)=\sum_{i=0}^{x-1} A[i]$. In this paper, we design a new succinct rank data structure with $r=n/(\log n)^{\Omega(t)}+n^{1-c}$ and query time $O(t)$ for some constant $c>0$, improving the previous best-known by Patrascu [Pat08], which has $r=n/(\frac{\log n}{t})^{\Omega(t)}+\tilde{O}(n^{3/4})$ bits of redundancy. For $r>n^{1-c}$, our space-time tradeoff matches the cell-probe lower bound by Patrascu and Viola [PV10], which asserts that $r$ must be at least $n/(\log n)^{O(t)}$. Moreover, one can avoid an $n^{1-c}$-bit lookup table when the data structure is implemented in the cell-probe model, achieving $r=\lceil n/(\log n)^{\Omega(t)}\rceil$. It matches the lower bound for the full range of parameters. En route to our new data structure design, we establish an interesting connection between succinct data structures and approximate nonnegative tensor decomposition. Our connection shows that for specific problems, to construct a space-efficient data structure, it suffices to approximate a particular tensor by a sum of (few) nonnegative rank-$1$ tensors. For the rank problem, we explicitly construct such an approximation, which yields an explicit construction of the data structure.