Optimal Oblivious Subspace Embeddings with Near-optimal Sparsity

TL;DR

This paper introduces an oblivious subspace embedding with optimal dimension and near-optimal sparsity, nearly matching the conjectured sparsity bounds and improving previous results, with extensions to faster matrix approximation and regression.

Contribution

It presents the first near-optimal sparsity oblivious subspace embedding matching the Nelson and Nguyen conjecture, and extends the approach to non-oblivious embeddings for improved computational efficiency.

Findings

Achieves optimal embedding dimension of Θ(d/ε²).

Attains near-optimal sparsity of Õ(1/ε) non-zero entries per column.

Provides faster algorithms for matrix approximation and regression tasks.

Abstract

An oblivious subspace embedding is a random $m\times n$ matrix $\Pi$ such that, for any $d$-dimensional subspace, with high probability $\Pi$ preserves the norms of all vectors in that subspace within a $1\pm\epsilon$ factor. In this work, we give an oblivious subspace embedding with the optimal dimension $m=\Theta(d/\epsilon^2)$ that has a near-optimal sparsity of $\tilde O(1/\epsilon)$ non-zero entries per column of $\Pi$. This is the first result to nearly match the conjecture of Nelson and Nguyen [FOCS 2013] in terms of the best sparsity attainable by an optimal oblivious subspace embedding, improving on a prior bound of $\tilde O(1/\epsilon^6)$ non-zeros per column [Chenakkod et al., STOC 2024]. We further extend our approach to the non-oblivious setting, proposing a new family of Leverage Score Sparsified embeddings with Independent Columns, which yield faster runtimes for matrix approximation and regression tasks. In our analysis, we develop a new method which uses a decoupling argument together with the cumulant method for bounding the edge universality error of isotropic random matrices. To achieve near-optimal sparsity, we combine this general-purpose approach with new traces inequalities that leverage the specific structure of our subspace embedding construction.