Accurate principal component analysis via a few iterations of alternating least squares

TL;DR

This paper demonstrates that a few iterations of alternating least squares with random initialization can efficiently produce near-optimal low-rank matrix approximations, eliminating the need for full convergence.

Contribution

It proves that a small number of ALS iterations suffice for nearly optimal low-rank approximation accuracy, simplifying and speeding up the process.

Findings

Few ALS iterations achieve near-optimal spectral and Frobenius norm accuracy

Convergence is unnecessary for high-quality approximations

Software can be improved by setting parameters for limited ALS iterations

Abstract

A few iterations of alternating least squares with a random starting point provably suffice to produce nearly optimal spectral- and Frobenius-norm accuracies of low-rank approximations to a matrix; iterating to convergence is unnecessary. Thus, software implementing alternating least squares can be retrofitted via appropriate setting of parameters to calculate nearly optimally accurate low-rank approximations highly efficiently, with no need for convergence.