Sparsity-Dimension Trade-Offs for Oblivious Subspace Embeddings

TL;DR

This paper establishes new lower bounds on the embedding dimension for oblivious subspace embeddings with specific sparsity levels, advancing understanding of the sparsity-dimension trade-offs in high-dimensional data embeddings.

Contribution

It provides the first lower bounds with multiplicative factors of d^2 and 1/ε for certain sparsity regimes, improving previous bounds and revealing fundamental trade-offs.

Findings

Lower bounds for embeddings with s ≤ 1/2.001ε nonzero entries per column.

Lower bounds for embeddings with s = Ω(log(1/ε)/ε) nonzero entries per column.

A general trade-off relation among d, ε, s, δ, and m.

Abstract

An oblivious subspace embedding (OSE), characterized by parameters $m,n,d,\epsilon,\delta$, is a random matrix $\Pi\in \mathbb{R}^{m\times n}$ such that for any $d$-dimensional subspace $T\subseteq \mathbb{R}^n$, $\Pr_\Pi[\forall x\in T, (1-\epsilon)\|x\|_2 \leq \|\Pi x\|_2\leq (1+\epsilon)\|x\|_2] \geq 1-\delta$. When an OSE has $s\le 1/2.001\epsilon$ nonzero entries in each column, we show it must hold that $m = \Omega\left(d^2/( \epsilon^2s^{1+O(\delta)})\right)$, which is the first lower bound with multiplicative factors of $d^2$ and $1/\epsilon$, improving on the previous $\Omega\left(d^2/s^{O(\delta)}\right)$ lower bound due to Li and Liu (PODS 2022). When an OSE has $s=\Omega(\log(1/\epsilon)/\epsilon)$ nonzero entries in each column, we show it must hold that $m = \Omega\left((d/\epsilon)^{1+1/4.001\epsilon s}/s^{O(\delta)}\right)$, which is the first lower bound with multiplicative factors of $d$ and $1/\epsilon$, improving on the previous $\Omega\left(d^{1+1/(16\epsilon s+4)}\right)$ lower bound due to Nelson and Nguyen (ICALP 2014). This second result is a special case of a more general trade-off among $d,\epsilon,s,\delta$ and $m$.