Low-Rank Matrix Approximations Do Not Need a Singular Value Gap

TL;DR

This paper demonstrates that low-rank matrix approximations are robust and well-posed without requiring a singular value gap, challenging the common assumption that such gaps are necessary for stability.

Contribution

It provides a systematic analysis showing the insensitivity of low-rank approximations to various perturbations, removing the need for a singular value gap for stability.

Findings

Low-rank approximations are insensitive to additive rank-preserving perturbations.

They remain stable under matrix perturbations that change the number of columns.

Connections between low-rank approximations and subspace angles are established when a singular value gap exists.

Abstract

This is a systematic investigation into the sensitivity of low-rank approximations of real matrices. We show that the low-rank approximation errors, in the two-norm, Frobenius norm and more generally, any Schatten p-norm, are insensitive to additive rank-preserving perturbations in the projector basis; and to matrix perturbations that are additive or change the number of columns (including multiplicative perturbations). Thus, low-rank matrix approximations are always well-posed and do not require a singular value gap. In the presence of a singular value gap, connections are established between low-rank approximations and subspace angles.