Linear Operator Approximate Message Passing (OpAMP)

TL;DR

This paper develops a new approximate message passing framework for dynamic data scenarios involving linear operators, with theoretical guarantees and applications to noisy rank-one spike estimation.

Contribution

Introduces a novel AMP framework with autoregressive memory for linear operators, including projection AMP, with rigorous Gaussian process approximation guarantees.

Findings

Theoretical state evolution for Gaussian matrices and non-separable denoisers.

Validation of the framework through numerical simulations.

Effective estimation of rank-one spikes with partial data updates.

Abstract

This paper introduces a framework for approximate message passing (AMP) in dynamic settings where the data at each iteration is passed through a linear operator. This framework is motivated in part by applications in large-scale, distributed computing where only a subset of the data is available at each iteration. An autoregressive memory term is used to mitigate information loss across iterations and a specialized algorithm, called projection AMP, is designed for the case where each linear operator is an orthogonal projection. Precise theoretical guarantees are provided for a class of Gaussian matrices and non-separable denoising functions. Specifically, it is shown that the iterates can be well-approximated in the high-dimensional limit by a Gaussian process whose second-order statistics are defined recursively via state evolution. These results are applied to the problem of estimating a rank-one spike corrupted by additive Gaussian noise using partial row updates, and the theory is validated by numerical simulations.