Randomly Initialized Alternating Least Squares: Fast Convergence for Matrix Sensing

TL;DR

This paper proves that the Alternating Least Squares method, when randomly initialized, converges efficiently to the true solution for rank-one matrix sensing problems with Gaussian measurements, requiring minimal samples.

Contribution

It establishes the first convergence guarantee for ALS with random initialization in matrix sensing, showing fast convergence with near-optimal sample complexity.

Findings

ALS with random initialization converges in O(log n + log(1/ε)) iterations.

Requires only a near-optimal number of Gaussian measurements.

Numerical experiments confirm theoretical results.

Abstract

We consider the problem of reconstructing rank-one matrices from random linear measurements, a task that appears in a variety of problems in signal processing, statistics, and machine learning. In this paper, we focus on the Alternating Least Squares (ALS) method. While this algorithm has been studied in a number of previous works, most of them only show convergence from an initialization close to the true solution and thus require a carefully designed initialization scheme. However, random initialization has often been preferred by practitioners as it is model-agnostic. In this paper, we show that ALS with random initialization converges to the true solution with $\varepsilon$-accuracy in $O(\log n + \log (1/\varepsilon)) $ iterations using only a near-optimal amount of samples, where we assume the measurement matrices to be i.i.d. Gaussian and where by $n$ we denote the ambient dimension. Key to our proof is the observation that the trajectory of the ALS iterates only depends very mildly on certain entries of the random measurement matrices. Numerical experiments corroborate our theoretical predictions.