Learning and Free Energies for Vector Approximate Message Passing

TL;DR

This paper introduces EM-VAMP, an algorithm combining vector approximate message passing with Expectation-Maximization to jointly recover signals and learn statistical parameters, demonstrating robustness and near-oracle performance.

Contribution

It extends VAMP by integrating EM, enabling joint signal recovery and parameter learning with a variational interpretation and improved robustness.

Findings

EM-VAMP performs well with ill-conditioned matrices.

Algorithm nearly matches oracle VAMP performance.

Provides a variational interpretation of the fixed points.

Abstract

Vector approximate message passing (VAMP) is a computationally simple approach to the recovery of a signal $\mathbf{x}$ from noisy linear measurements $\mathbf{y}=\mathbf{Ax}+\mathbf{w}$. Like the AMP proposed by Donoho, Maleki, and Montanari in 2009, VAMP is characterized by a rigorous state evolution (SE) that holds under certain large random matrices and that matches the replica prediction of optimality. But while AMP's SE holds only for large i.i.d. sub-Gaussian $\mathbf{A}$, VAMP's SE holds under the much larger class: right-rotationally invariant $\mathbf{A}$. To run VAMP, however, one must specify the statistical parameters of the signal and noise. This work combines VAMP with Expectation-Maximization to yield an algorithm, EM-VAMP, that can jointly recover $\mathbf{x}$ while learning those statistical parameters. The fixed points of the proposed EM-VAMP algorithm are shown to be stationary points of a certain constrained free-energy, providing a variational interpretation of the algorithm. Numerical simulations show that EM-VAMP is robust to highly ill-conditioned $\mathbf{A}$ with performance nearly matching oracle-parameter VAMP.