Stein Variational Message Passing for Continuous Graphical Models

TL;DR

This paper introduces a distributed inference algorithm for continuous graphical models that extends Stein variational gradient descent by incorporating local kernels based on the Markov blanket, improving efficiency and scalability.

Contribution

It develops a novel distributed inference method combining SVGD with local kernels tailored to the Markov structure, enabling decentralized inference with theoretical guarantees.

Findings

Outperforms standard MCMC and particle message passing methods

Alleviates high-dimensionality issues in graphical models

Provides theoretical analysis of local kernel approximation

Abstract

We propose a novel distributed inference algorithm for continuous graphical models, by extending Stein variational gradient descent (SVGD) to leverage the Markov dependency structure of the distribution of interest. Our approach combines SVGD with a set of structured local kernel functions defined on the Markov blanket of each node, which alleviates the curse of high dimensionality and simultaneously yields a distributed algorithm for decentralized inference tasks. We justify our method with theoretical analysis and show that the use of local kernels can be viewed as a new type of localized approximation that matches the target distribution on the conditional distributions of each node over its Markov blanket. Our empirical results show that our method outperforms a variety of baselines including standard MCMC and particle message passing methods.