Warm-Starting in Message Passing algorithms

TL;DR

This paper introduces scalable warm-started methods that approximate the LMMSE estimator within VAMP, enabling efficient large-scale linear inverse problem solving with convergence guarantees.

Contribution

It proposes a class of warm-started message passing algorithms that approximate VAMP, including Onsager correction and State Evolution, and relates to Memory AMP.

Findings

Converges to VAMP fixed point with reduced complexity

Provides Onsager correction and State Evolution for the new methods

Shows Memory AMP as a special case of the proposed class

Abstract

Vector Approximate Message Passing (VAMP) provides the means of solving a linear inverse problem in a Bayes-optimal way assuming the measurement operator is sufficiently random. However, VAMP requires implementing the linear minimum mean squared error (LMMSE) estimator at every iteration, which makes the algorithm intractable for large-scale problems. In this work, we present a class of warm-started (WS) methods that provides a scalable approximation of LMMSE within VAMP. We show that a Message Passing (MP) algorithm equipped with a method from this class can converge to the fixed point of VAMP while having a per-iteration computational complexity proportional to that of AMP. Additionally, we provide the Onsager correction and a multi-dimensional State Evolution for MP utilizing one of the WS methods. Lastly, we show that the approximation approach used in the recently proposed Memory AMP (MAMP) algorithm is a special case of the developed class of WS methods.