A provably convergent alternating minimization method for mean field inference

TL;DR

This paper introduces a modified mean-field inference algorithm with added penalization to ensure convergence to a critical point, providing theoretical guarantees and maintaining simple update steps.

Contribution

It proposes a provably convergent alternating minimization method for mean-field inference by incorporating a penalization term, addressing previous convergence uncertainties.

Findings

Guarantees convergence to a critical point

Maintains closed-form updates at each step

Provides convergence rate estimates

Abstract

Mean-Field is an efficient way to approximate a posterior distribution in complex graphical models and constitutes the most popular class of Bayesian variational approximation methods. In most applications, the mean field distribution parameters are computed using an alternate coordinate minimization. However, the convergence properties of this algorithm remain unclear. In this paper, we show how, by adding an appropriate penalization term, we can guarantee convergence to a critical point, while keeping a closed form update at each step. A convergence rate estimate can also be derived based on recent results in non-convex optimization.