Concentration of the Frobenius norm of generalized matrix inverses

TL;DR

This paper derives finite-size concentration bounds for the Frobenius norm of generalized matrix inverses, especially for sparse and Moore-Penrose pseudoinverses, using convex Gaussian min-max theorem techniques.

Contribution

It provides the first finite-size concentration bounds for the Frobenius norm of $\, ext{ell}^p$-minimal generalized inverses of Gaussian matrices, including sparse pseudoinverses.

Findings

Exponential concentration bound for the sparse pseudoinverse at p=1.

Finite-size concentration bounds for the Moore-Penrose pseudoinverse at p=2.

Application of convex Gaussian min-max theorem to derive non-asymptotic bounds.

Abstract

In many applications it is useful to replace the Moore-Penrose pseudoinverse (MPP) by a different generalized inverse with more favorable properties. We may want, for example, to have many zero entries, but without giving up too much of the stability of the MPP. One way to quantify stability is by how much the Frobenius norm of a generalized inverse exceeds that of the MPP. In this paper we derive finite-size concentration bounds for the Frobenius norm of $\ell^p$-minimal general inverses of iid Gaussian matrices, with $1 \leq p \leq 2$. For $p = 1$ we prove exponential concentration of the Frobenius norm of the sparse pseudoinverse; for $p = 2$, we get a similar concentration bound for the MPP. Our proof is based on the convex Gaussian min-max theorem, but unlike previous applications which give asymptotic results, we derive finite-size bounds.