Lower Bounds for Sparse Oblivious Subspace Embeddings

TL;DR

This paper establishes fundamental lower bounds on the number of rows needed for sparse oblivious subspace embeddings, confirming the optimality of Count-Sketch and improving bounds for certain sparsity levels.

Contribution

It proves new lower bounds on the embedding dimension for sparse OSEs, demonstrating optimality of Count-Sketch and enhancing previous bounds for specific sparsity.

Findings

Count-Sketch is optimal for one nonzero per column.

Lower bound of m = Ω(d^2/(psilon^2 elta)) for certain sparse OSEs.

Improved lower bound of m = Ω(psilon^{O(elta)} d^2) for embeddings with /(9psilon) nonzeros per column.

Abstract

An oblivious subspace embedding (OSE), characterized by parameters $m,n,d,\epsilon,\delta$, is a random matrix $\Pi\in \mathbb{R}^{m\times n}$ such that for any $d$-dimensional subspace $T\subseteq \mathbb{R}^n$, $\Pr_\Pi[\forall x\in T, (1-\epsilon)\|x\|_2 \leq \|\Pi x\|_2\leq (1+\epsilon)\|x\|_2] \geq 1-\delta$. For $\epsilon$ and $\delta$ at most a small constant, we show that any OSE with one nonzero entry in each column must satisfy that $m = \Omega(d^2/(\epsilon^2\delta))$, establishing the optimality of the classical Count-Sketch matrix. When an OSE has $1/(9\epsilon)$ nonzero entries in each column, we show it must hold that $m = \Omega(\epsilon^{O(\delta)} d^2)$, improving on the previous $\Omega(\epsilon^2 d^2)$ lower bound due to Nelson and Nguyen (ICALP 2014).