An Overview of Multi-Processor Approximate Message Passing

TL;DR

This paper reviews multi-processor variants of approximate message passing algorithms, discussing their architectures, advantages, limitations, recent theoretical developments, and techniques to reduce communication costs in large-scale signal reconstruction tasks.

Contribution

It provides a comprehensive overview of two MP-AMP variants, analyzing their design, recent research progress, and practical considerations for large-scale applications.

Findings

State evolution results for MP-AMP algorithms

Comparison of row-MP-AMP and column-MP-AMP approaches

Use of data compression to reduce communication overhead

Abstract

Approximate message passing (AMP) is an algorithmic framework for solving linear inverse problems from noisy measurements, with exciting applications such as reconstructing images, audio, hyper spectral images, and various other signals, including those acquired in compressive signal acquisiton systems. The growing prevalence of big data systems has increased interest in large-scale problems, which may involve huge measurement matrices that are unsuitable for conventional computing systems. To address the challenge of large-scale processing, multiprocessor (MP) versions of AMP have been developed. We provide an overview of two such MP-AMP variants. In row-MP-AMP, each computing node stores a subset of the rows of the matrix and processes corresponding measurements. In column- MP-AMP, each node stores a subset of columns, and is solely responsible for reconstructing a portion of the signal. We will discuss pros and cons of both approaches, summarize recent research results for each, and explain when each one may be a viable approach. Aspects that are highlighted include some recent results on state evolution for both MP-AMP algorithms, and the use of data compression to reduce communication in the MP network.