Iterative Collaborative Filtering for Sparse Matrix Estimation

TL;DR

This paper introduces an iterative collaborative filtering algorithm for sparse matrix estimation, demonstrating that it achieves vanishing error under certain sampling conditions and robustness to noise and model mis-specification.

Contribution

The paper proposes a novel iterative variant of collaborative filtering tailored for sparse observations and provides theoretical guarantees on its estimation error decay.

Findings

Estimation error decays to zero as matrix size increases under specified sampling conditions.

Algorithm is robust to arbitrary bounded noise in observations.

Performance extends to approximately low-rank matrices, not just exactly low-rank.

Abstract

We consider sparse matrix estimation where the goal is to estimate an $n\times n$ matrix from noisy observations of a small subset of its entries. We analyze the estimation error of the popularly utilized collaborative filtering algorithm for the sparse regime. Specifically, we propose a novel iterative variant of the algorithm, adapted to handle the setting of sparse observations. We establish that as long as the fraction of entries observed at random scale as $\frac{\log^{1+\kappa}(n)}{n}$ for any fixed $\kappa > 0$, the estimation error with respect to the $\max$-norm decays to $0$ as $n\to\infty$ assuming the underlying matrix of interest has constant rank $r$. Our result is robust to model mis-specification in that if the underlying matrix is approximately rank $r$, then the estimation error decays to the approximate error with respect to the $\max$-norm. In the process, we establish algorithm's ability to handle arbitrary bounded noise in the observations.