Learning Mixtures of Low-Rank Models

TL;DR

This paper introduces a three-stage algorithm for learning mixtures of low-rank models, effectively reconstructing multiple matrices from unlabelled measurements with guarantees on accuracy, efficiency, and noise stability.

Contribution

It proposes a novel meta-algorithm that addresses non-convexity in mixture low-rank models, achieving near-optimal sample and computational complexity with theoretical guarantees.

Findings

Algorithm recovers unknown matrices with near-optimal sample complexity.

The method is stable against random noise.

Empirical results confirm theoretical guarantees.

Abstract

We study the problem of learning mixtures of low-rank models, i.e. reconstructing multiple low-rank matrices from unlabelled linear measurements of each. This problem enriches two widely studied settings -- low-rank matrix sensing and mixed linear regression -- by bringing latent variables (i.e. unknown labels) and structural priors (i.e. low-rank structures) into consideration. To cope with the non-convexity issues arising from unlabelled heterogeneous data and low-complexity structure, we develop a three-stage meta-algorithm that is guaranteed to recover the unknown matrices with near-optimal sample and computational complexities under Gaussian designs. In addition, the proposed algorithm is provably stable against random noise. We complement the theoretical studies with empirical evidence that confirms the efficacy of our algorithm.