Reactive Message Passing for Scalable Bayesian Inference

TL;DR

Reactive Message Passing (RMP) is a new framework for scalable, robust Bayesian inference that operates without fixed schedules, using reactive programming principles to efficiently handle large probabilistic models.

Contribution

The paper introduces RMP as a novel, schedule-free message passing framework and provides ReactiveMP.jl, a Julia package implementing various inference algorithms based on RMP.

Findings

RMP improves robustness and scalability of message passing inference.

ReactiveMP.jl efficiently handles large-scale models with hundreds of thousands of variables.

Experimental results show superior performance over existing Julia packages.

Abstract

We introduce Reactive Message Passing (RMP) as a framework for executing schedule-free, robust and scalable message passing-based inference in a factor graph representation of a probabilistic model. RMP is based on the reactive programming style that only describes how nodes in a factor graph react to changes in connected nodes. The absence of a fixed message passing schedule improves robustness, scalability and execution time of the inference procedure. We also present ReactiveMP.jl, which is a Julia package for realizing RMP through minimization of a constrained Bethe free energy. By user-defined specification of local form and factorization constraints on the variational posterior distribution, ReactiveMP.jl executes hybrid message passing algorithms including belief propagation, variational message passing, expectation propagation, and expectation maximisation update rules. Experimental results demonstrate the improved performance of ReactiveMP-based RMP in comparison to other Julia packages for Bayesian inference across a range of probabilistic models. In particular, we show that the RMP framework is able to run Bayesian inference for large-scale probabilistic state space models with hundreds of thousands of random variables on a standard laptop computer.