Performance Trade-Offs in Multi-Processor Approximate Message Passing

TL;DR

This paper investigates the trade-offs between computation, communication, and accuracy in multi-processor approximate message passing for large-scale linear inverse problems, framing it as a multi-objective optimization problem and analyzing its properties.

Contribution

It formulates the cost trade-offs in multi-processor AMP as a convex multi-objective optimization problem and characterizes the Pareto optimality of these trade-offs.

Findings

Achievable trade-off region is convex.

Numerical verification of theoretical properties.

Conjecture on cost scaling with mean squared error.

Abstract

We consider large-scale linear inverse problems in Bayesian settings. Our general approach follows a recent line of work that applies the approximate message passing (AMP) framework in multi-processor (MP) computational systems by storing and processing a subset of rows of the measurement matrix along with corresponding measurements at each MP node. In each MP-AMP iteration, nodes of the MP system and its fusion center exchange lossily compressed messages pertaining to their estimates of the input. There is a trade-off between the physical costs of the reconstruction process including computation time, communication loads, and the reconstruction quality, and it is impossible to simultaneously minimize all the costs. We pose this minimization as a multi-objective optimization problem (MOP), and study the properties of the best trade-offs (Pareto optimality) in this MOP. We prove that the achievable region of this MOP is convex, and conjecture how the combined cost of computation and communication scales with the desired mean squared error. These properties are verified numerically.