Probabilistic Inference and Learning with Stein's Method

TL;DR

This paper offers a comprehensive overview of Stein's method for probabilistic inference and learning, detailing constructions, properties, and connections to variational gradient descent, with rigorous theoretical foundations.

Contribution

It introduces systematic recipes for Stein discrepancy construction and explores their properties, linking Stein operators to Stein variational gradient descent in a rigorous manner.

Findings

Provides detailed recipes for Stein discrepancy construction

Analyzes properties like computability and convergence detection

Establishes connections between Stein operators and gradient descent

Abstract

This monograph provides a rigorous overview of theoretical and methodological aspects of probabilistic inference and learning with Stein's method. Recipes are provided for constructing Stein discrepancies from Stein operators and Stein sets, and properties of these discrepancies such as computability, separation, convergence detection, and convergence control are discussed. Further, the connection between Stein operators and Stein variational gradient descent is set out in detail. The main definitions and results are precisely stated, and references to all proofs are provided.