Best low-rank approximations and Kolmogorov n-widths

TL;DR

This paper explores the relationship between best low-rank matrix approximations in spectral norm and Kolmogorov n-widths, providing explicit constructions and alternative solutions to SVD-based methods.

Contribution

It characterizes all optimal spaces for low-rank approximation and introduces a method to generate multiple optimal solutions beyond SVD.

Findings

Characterization of all optimal spaces for low-rank approximation.

Explicit construction of sequences of optimal spaces.

Alternative solutions to truncated SVD with problem-specific properties.

Abstract

We relate the problem of best low-rank approximation in the spectral norm for a matrix $A$ to Kolmogorov $n$-widths and corresponding optimal spaces. We characterize all the optimal spaces for the image of the Euclidean unit ball under $A$ and we show that any orthonormal basis in an $n$-dimensional optimal space generates a best rank-$n$ approximation to $A$. We also present a simple and explicit construction to obtain a sequence of optimal $n$-dimensional spaces once an initial optimal space is known. This results in a variety of solutions to the best low-rank approximation problem and provides alternatives to the truncated singular value decomposition. This variety can be exploited to obtain best low-rank approximations with problem-oriented properties.