# A Simple Derivation of AMP and its State Evolution via First-Order Cancellation

## TL;DR

This paper offers a heuristic, simplified derivation of the AMP algorithm and its state evolution for linear regression, providing new insights into why AMP works well with large i.i.d. matrices.

## Contribution

It introduces a first-order cancellation approach to derive AMP and its state evolution, simplifying understanding beyond traditional belief propagation methods.

## Key findings

- Heuristic derivation of AMP using first-order cancellation
- Insights into the importance of large i.i.d. matrices for AMP
- Simplified explanation of the state evolution formalism

## Abstract

We consider the linear regression problem, where the goal is to recover the vector $\boldsymbol{x}\in\mathbb{R}^n$ from measurements $\boldsymbol{y}=\boldsymbol{A}\boldsymbol{x}+\boldsymbol{w}\in\mathbb{R}^m$ under known matrix $\boldsymbol{A}$ and unknown noise $\boldsymbol{w}$. For large i.i.d. sub-Gaussian $\boldsymbol{A}$, the approximate message passing (AMP) algorithm is precisely analyzable through a state-evolution (SE) formalism, which furthermore shows that AMP is Bayes optimal in certain regimes. The rigorous SE proof, however, is long and complicated. And, although the AMP algorithm can be derived as an approximation of loop belief propagation (LBP), this viewpoint provides little insight into why large i.i.d. $\boldsymbol{A}$ matrices are important for AMP, and why AMP has a state evolution. In this work, we provide a heuristic derivation of AMP and its state evolution, based on the idea of "first-order cancellation," that provides insights missing from the LBP derivation while being much shorter than the rigorous SE proof.