Principled Parallel Mean-Field Inference for Discrete Random Fields

TL;DR

This paper introduces a new parallelizable proximal gradient method for mean-field inference in discrete random fields, offering faster convergence, better optima, and reduced parameter sensitivity compared to traditional techniques.

Contribution

A novel proximal gradient-based approach for mean-field inference that is naturally parallelizable, converges reliably, and outperforms existing methods in speed and quality.

Findings

Faster convergence than traditional mean-field methods

Often finds better optima in practice

Less sensitive to parameter choices

Abstract

Mean-field variational inference is one of the most popular approaches to inference in discrete random fields. Standard mean-field optimization is based on coordinate descent and in many situations can be impractical. Thus, in practice, various parallel techniques are used, which either rely on ad-hoc smoothing with heuristically set parameters, or put strong constraints on the type of models. In this paper, we propose a novel proximal gradient-based approach to optimizing the variational objective. It is naturally parallelizable and easy to implement. We prove its convergence, and then demonstrate that, in practice, it yields faster convergence and often finds better optima than more traditional mean-field optimization techniques. Moreover, our method is less sensitive to the choice of parameters.