Orthogonal Approximate Message Passing Algorithms for Rectangular Spiked Matrix Models with Rotationally Invariant Noise

TL;DR

This paper introduces an orthogonal approximate message passing algorithm for rectangular spiked matrix models with rotationally invariant noise, providing a rigorous analysis and an optimal variant that potentially achieves Bayes-optimal performance.

Contribution

The paper develops a new OAMP algorithm for RI noise models, establishes exact state evolution, and derives an optimal version minimizing mean-squared error, extending AMP theory.

Findings

Exact state evolution characterizes high-dimensional dynamics.

Optimal OAMP minimizes mean-squared error at each iteration.

Conjecture of statistical and Bayes-optimality for general RI noise.

Abstract

We propose an orthogonal approximate message passing (OAMP) algorithm for signal estimation in the rectangular spiked matrix model with general rotationally invariant (RI) noise. We establish a rigorous state evolution that exactly characterizes the high-dimensional dynamics of the algorithm. Building on this framework, we derive an optimal variant of OAMP that minimizes the predicted mean-squared error at each iteration. For the special case of i.i.d. Gaussian noise, the fixed point of the proposed OAMP algorithm coincides with that of the standard AMP algorithm. For general RI noise models, we conjecture that the optimal OAMP algorithm is statistically optimal within a broad class of iterative methods, and achieves Bayes-optimal performance in certain regimes.