Perturbation expansions and error bounds for the truncated singular value decomposition

TL;DR

This paper develops new perturbation expansions and error bounds for the truncated singular value decomposition (TSVD), providing insights into its stability and accuracy under perturbations, with applications to matrix denoising.

Contribution

It introduces the first perturbation expansions and error bounds for TSVD, including second-order expansions and a universal error bound independent of subspace sensitivity.

Findings

First-order perturbation expansion for TSVD.

Second-order expansion for rank-$r$ matrices.

Universal error bound depending on smallest singular value.

Abstract

Truncated singular value decomposition is a reduced version of the singular value decomposition in which only a few largest singular values are retained. This paper presents a novel perturbation analysis for the truncated singular value decomposition for real matrices. First, we describe perturbation expansions for the singular value truncation of order $r$. We extend perturbation results for the singular subspace decomposition to derive the first-order perturbation expansion of the truncated operator about a matrix with rank greater than or equal to $r$. Observing that the first-order expansion can be greatly simplified when the matrix has exact rank $r$, we further show that the singular value truncation admits a simple second-order perturbation expansion about a rank-$r$ matrix. Second, we introduce the first-known error bound on the linear approximation of the truncated singular value decomposition of a perturbed rank-$r$ matrix. Our bound only depends on the least singular value of the unperturbed matrix and the norm of the perturbation matrix. Intriguingly, while the singular subspaces are known to be extremely sensitive to additive noises, the newly established error bound holds universally for perturbations with arbitrary magnitude. Finally, we demonstrate an application of our results to the analysis of the mean squared error associated with the TSVD-based matrix denoising solution.