Preserving Extreme Singular Values with One Oblivious Sketch

TL;DR

This paper investigates the ability of a single linear sketch to preserve the extreme singular values of all rank-$r$ matrices, proposing new bounds, constructions, and demonstrating practical benefits and limitations through experiments.

Contribution

It formalizes a conjecture that $O(r \, \log r)$ sketch size suffices for singular value preservation and introduces a combined sketching method that improves conditioning.

Findings

A combined sketching method collapses singular values to a common scale.

Balancing improves conditioning and accelerates iterative solvers.

Any sketch achieving accurate singular values must have size at least proportional to $r/\varepsilon^2$.

Abstract

We study when a single linear sketch can control the largest and smallest nonzero singular values of every rank-$r$ matrix. Classical oblivious embeddings require $s=\Theta(r/\varepsilon^{2})$ for $(1\pm\varepsilon)$ distortion, but this does not yield constant-factor control of extreme singular values or condition numbers. We formalize a conjecture that $s=O(r\log r)$ suffices for such preservation. On the constructive side, we show that combining a sparse oblivious sketch with a deterministic geometric balancing map produces a sketch whose nonzero singular values collapse to a common scale under bounded condition number and coherence. On the negative side, we prove that any oblivious sketch achieving relative $\varepsilon$-accurate singular values for all rank-$r$ matrices must satisfy $s=\Omega((r+\log(1/\delta))/\varepsilon^{2})$. Numerical experiments on structured matrix families confirm that balancing improves conditioning and accelerates iterative solvers, while coherent or nearly rank-deficient inputs manifest the predicted failure modes.