# Mutual Information and Optimality of Approximate Message-Passing in Random Linear Estimation

## TL;DR

This paper rigorously establishes the mutual information and optimality of approximate message-passing algorithms in Gaussian linear estimation, showing they achieve minimal mean-square error in relevant scenarios.

## Contribution

It provides a rigorous proof that the replica formula bounds the mutual information and demonstrates the optimality of AMP algorithms outside the hard phase.

## Key findings

- Replica formula bounds mutual information from above.
- AMP algorithms reach minimal MSE outside the hard phase.
- No algorithmically hard phase exists in these systems.

## Abstract

We consider the estimation of a signal from the knowledge of its noisy linear random Gaussian projections. A few examples where this problem is relevant are compressed sensing, sparse superposition codes, and code division multiple access. There has been a number of works considering the mutual information for this problem using the replica method from statistical physics. Here we put these considerations on a firm rigorous basis. First, we show, using a Guerra-Toninelli type interpolation, that the replica formula yields an upper bound to the exact mutual information. Secondly, for many relevant practical cases, we present a converse lower bound via a method that uses spatial coupling, state evolution analysis and the I-MMSE theorem. This yields a single letter formula for the mutual information and the minimal-mean-square error for random Gaussian linear estimation of all discrete bounded signals. In addition, we prove that the low complexity approximate message-passing algorithm is optimal outside of the so-called hard phase, in the sense that it asymptotically reaches the minimal-mean-square error. In this work spatial coupling is used primarily as a proof technique. However our results also prove two important features of spatially coupled noisy linear random Gaussian estimation. First there is no algorithmically hard phase. This means that for such systems approximate message-passing always reaches the minimal-mean-square error. Secondly, in a proper limit the mutual information associated to such systems is the same as the one of uncoupled linear random Gaussian estimation.