On Recovering the Best Rank-r Approximation from Few Entries

TL;DR

This paper explores methods to accurately recover the best low-rank approximation of a large matrix from limited entries, demonstrating that the second approach outperforms spectral truncation in most cases.

Contribution

It introduces and compares two agnostic methods for low-rank approximation from few entries, highlighting the superiority of projected gradient descent.

Findings

Projected gradient descent outperforms spectral truncation.

Error depends on matrix's proximity to low rank.

Both theoretical and numerical results support effectiveness.

Abstract

In this note, we investigate how well we can reconstruct the best rank-$r$ approximation of a large matrix from a small number of its entries. We show that even if a data matrix is of full rank and cannot be approximated well by a low-rank matrix, its best low-rank approximations may still be reliably computed or estimated from a small number of its entries. This is especially relevant from a statistical viewpoint: the best low-rank approximations to a data matrix are often of more interest than itself because they capture the more stable and oftentimes more reproducible properties of an otherwise complicated data-generating model. In particular, we investigate two agnostic approaches: the first is based on spectral truncation; and the second is a projected gradient descent based optimization procedure. We argue that, while the first approach is intuitive and reasonably effective, the latter has far superior performance in general. We show that the error depends on how close the matrix is to being of low rank. Both theoretical and numerical evidence is presented to demonstrate the effectiveness of the proposed approaches.