Theory and computation for structured variational inference

TL;DR

This paper advances the theoretical understanding and computational methods for star-structured variational inference, providing existence, uniqueness, error bounds, and a gradient-based algorithm with guarantees, applicable to various Bayesian models.

Contribution

It establishes the first theoretical results for star-structured variational inference, including error bounds and a provably convergent algorithm, extending prior mean-field work.

Findings

Proved existence, uniqueness, and self-consistency of star-structured variational approximations.

Derived quantitative error bounds for the approximation to the true posterior.

Developed a gradient-based algorithm with provable guarantees for computation.

Abstract

Structured variational inference constitutes a core methodology in modern statistical applications. Unlike mean-field variational inference, the approximate posterior is assumed to have interdependent structure. We consider the natural setting of star-structured variational inference, where a root variable impacts all the other ones. We prove the first results for existence, uniqueness, and self-consistency of the variational approximation. In turn, we derive quantitative approximation error bounds for the variational approximation to the posterior, extending prior work from the mean-field setting to the star-structured setting. We also develop a gradient-based algorithm with provable guarantees for computing the variational approximation using ideas from optimal transport theory. We explore the implications of our results for Gaussian measures and hierarchical Bayesian models, including generalized linear models with location family priors and spike-and-slab priors with one-dimensional debiasing. As a by-product of our analysis, we develop new stability results for star-separable transport maps which might be of independent interest.