On Orthogonal Approximate Message Passing

TL;DR

This paper explores Orthogonal Approximate Message Passing (OAMP), demonstrating its generality, the importance of orthogonality for Gaussian error distribution, and unifying various AMP algorithms under this principle.

Contribution

It introduces a Gram-Schmidt orthogonalization procedure for OAMP, proving its broad applicability and unifying different AMP-type algorithms through the orthogonal principle.

Findings

OAMP achieves asymptotic Gaussianity of errors without the Onsager term.

The Gram-Schmidt orthogonalization method is versatile for various algorithms.

Unification of AMP, EP, turbo, and OAMP under the orthogonal principle.

Abstract

Approximate Message Passing (AMP) is an efficient iterative parameter-estimation technique for certain high-dimensional linear systems with non-Gaussian distributions, such as sparse systems. In AMP, a so-called Onsager term is added to keep estimation errors approximately Gaussian. Orthogonal AMP (OAMP) does not require this Onsager term, relying instead on an orthogonalization procedure to keep the current errors uncorrelated with (i.e., orthogonal to) past errors. \LL{In this paper, we show the generality and significance of the orthogonality in ensuring that errors are "asymptotically independently and identically distributed Gaussian" (AIIDG).} This AIIDG property, which is essential for the attractive performance of OAMP, holds for separable functions. \LL{We present a simple and versatile procedure to establish the orthogonality through Gram-Schmidt (GS) orthogonalization, which is applicable to any prototype. We show that different AMP-type algorithms, such as expectation propagation (EP), turbo, AMP and OAMP, can be unified under the orthogonal principle.} The simplicity and generality of OAMP provide efficient solutions for estimation problems beyond the classical linear models. \LL{As an example, we study the optimization of OAMP via the GS model and GS orthogonalization.} More related applications will be discussed in a companion paper where new algorithms are developed for problems with multiple constraints and multiple measurement variables.