On the smoothed analysis of the smallest singular value with discrete noise

TL;DR

This paper analyzes the smallest singular value of a matrix perturbed by discrete noise, extending previous results to matrices with only some singular values bounded and establishing bounds on the decay rate of small singular values.

Contribution

It extends existing bounds on the smallest singular value to matrices with only some bounded singular values and establishes fundamental limits on the decay rate of small singular values under discrete noise.

Findings

Probability bound for smallest singular value with relaxed conditions

Extension of Rudelson-Vershynin result to matrices with some large singular values

Lower bound on the decay rate of small singular values in noisy matrices

Abstract

Let $A$ be an $n\times n$ real matrix, and let $M$ be an $n\times n$ random matrix whose entries are i.i.d sub-Gaussian random variables with mean $0$ and variance $1$. We make two contributions to the study of $s_n(A+M)$, the smallest singular value of $A+M$. (1) We show that for all $\epsilon \geq 0$, $$\mathbb{P}[s_n(A + M) \leq \epsilon] = O(\epsilon \sqrt{n}) + 2e^{-\Omega(n)},$$ provided only that $A$ has $\Omega (n)$ singular values which are $O(\sqrt{n})$. This extends a well-known result of Rudelson and Vershynin, which requires all singular values of $A$ to be $O(\sqrt{n})$. (2) We show that any bound of the form $$\sup_{\|{A}\|\leq n^{C_1}}\mathbb{P}[s_n(A+M)\leq n^{-C_3}] \leq n^{-C_2}$$ must have $C_3 = \Omega (C_1 \sqrt{C_2})$. This complements a result of Tao and Vu, who proved such a bound with $C_3 = O(C_1C_2 + C_1 + 1)$, and counters their speculation of possibly taking $C_3 = O(C_1 + C_2)$.