Gaussian Belief Propagation: Theory and Aplication

TL;DR

This paper develops a distributed Gaussian belief propagation algorithm for solving linear systems efficiently without matrix inversion, demonstrating its effectiveness in large-scale network applications through extensive simulations.

Contribution

It introduces a novel GaBP algorithm with convergence guarantees for arbitrary matrices and applies it to large-scale network problems, showcasing its scalability and efficiency.

Findings

Converges for arbitrary matrices with a new convergence method

Efficient in large-scale networks with millions of nodes

Demonstrated scalability on supercomputers with extensive simulations

Abstract

The canonical problem of solving a system of linear equations arises in numerous contexts in information theory, communication theory, and related fields. In this contribution, we develop a solution based upon Gaussian belief propagation (GaBP) that does not involve direct matrix inversion. The iterative nature of our approach allows for a distributed message-passing implementation of the solution algorithm. In the first part of this thesis, we address the properties of the GaBP solver. We characterize the rate of convergence, enhance its message-passing efficiency by introducing a broadcast version, discuss its relation to classical solution methods including numerical examples. We present a new method for forcing the GaBP algorithm to converge to the correct solution for arbitrary column dependent matrices. In the second part we give five applications to illustrate the applicability of the GaBP algorithm to very large computer networks: Peer-to-Peer rating, linear detection, distributed computation of support vector regression, efficient computation of Kalman filter and distributed linear programming. Using extensive simulations on up to 1,024 CPUs in parallel using IBM Bluegene supercomputer we demonstrate the attractiveness and applicability of the GaBP algorithm, using real network topologies with up to millions of nodes and hundreds of millions of communication links. We further relate to several other algorithms and explore their connection to the GaBP algorithm.