Multiprocessor Approximate Message Passing with Column-Wise Partitioning

TL;DR

This paper extends approximate message passing (AMP) algorithms to multiprocessor systems with column-wise data partitioning, providing theoretical analysis and demonstrating convergence properties through state evolution, with practical damping improvements.

Contribution

It introduces a column-wise multiprocessor AMP (C-MP-AMP) framework with theoretical state evolution analysis, showing convergence guarantees and performance insights.

Findings

C-MP-AMP obeys state evolution under similar conditions as AMP.

The state evolution of C-MP-AMP converges to a performance no worse than AMP.

Damping improves convergence in scenarios outside theoretical guarantees.

Abstract

Solving a large-scale regularized linear inverse problem using multiple processors is important in various real-world applications due to the limitations of individual processors and constraints on data sharing policies. This paper focuses on the setting where the matrix is partitioned column-wise. We extend the algorithmic framework and the theoretical analysis of approximate message passing (AMP), an iterative algorithm for solving linear inverse problems, whose asymptotic dynamics are characterized by state evolution (SE). In particular, we show that column-wise multiprocessor AMP (C-MP-AMP) obeys an SE under the same assumptions when the SE for AMP holds. The SE results imply that (i) the SE of C-MP-AMP converges to a state that is no worse than that of AMP and (ii) the asymptotic dynamics of C-MP-AMP and AMP can be identical. Moreover, for a setting that is not covered by SE, numerical results show that damping can improve the convergence performance of C-MP-AMP.