Optimal Subspace Embeddings: Resolving Nelson-Nguyen Conjecture Up to Sub-Polylogarithmic Factors

TL;DR

This paper proves a conjecture on the optimal dimension and sparsity of oblivious subspace embeddings, providing near-optimal parameters and a new matrix multiplication time reduction for linear regression tasks.

Contribution

It resolves the Nelson-Nguyen conjecture up to sub-polylogarithmic factors and introduces a novel matrix concentration technique called iterative decoupling.

Findings

Constructs a random matrix with near-optimal embedding properties.

Achieves the fastest known sub-current matrix multiplication time for certain linear regression problems.

Introduces iterative decoupling for refined matrix concentration bounds.

Abstract

We give a proof of the conjecture of Nelson and Nguyen [FOCS 2013] on the optimal dimension and sparsity of oblivious subspace embeddings, up to sub-polylogarithmic factors: For any $n\geq d$ and $\epsilon\geq d^{-O(1)}$, there is a random $\tilde O(d/\epsilon^2)\times n$ matrix $\Pi$ with $\tilde O(\log(d)/\epsilon)$ non-zeros per column such that for any $A\in\mathbb{R}^{n\times d}$, with high probability, $(1-\epsilon)\|Ax\|\leq\|\Pi Ax\|\leq(1+\epsilon)\|Ax\|$ for all $x\in\mathbb{R}^d$, where $\tilde O(\cdot)$ hides only sub-polylogarithmic factors in $d$. Our result in particular implies a new fastest sub-current matrix multiplication time reduction of size $\tilde O(d/\epsilon^2)$ for a broad class of $n\times d$ linear regression tasks. A key novelty in our analysis is a matrix concentration technique we call iterative decoupling, which we use to fine-tune the higher-order trace moment bounds attainable via existing random matrix universality tools [Brailovskaya and van Handel, GAFA 2024].