Multi-Modal Mean-Fields via Cardinality-Based Clamping

TL;DR

This paper introduces a novel mixture-based Mean Field inference method that better captures variable dependencies in probabilistic models, improving approximation quality in computer vision tasks.

Contribution

It proposes a mixture of Mean Field distributions conditioned on variable groups, utilizing a temperature parameter to enhance posterior approximation efficiency.

Findings

Improved posterior approximation by mixture models.

Enhanced performance in real-world computer vision algorithms.

Efficient inference through temperature-based grouping.

Abstract

Mean Field inference is central to statistical physics. It has attracted much interest in the Computer Vision community to efficiently solve problems expressible in terms of large Conditional Random Fields. However, since it models the posterior probability distribution as a product of marginal probabilities, it may fail to properly account for important dependencies between variables. We therefore replace the fully factorized distribution of Mean Field by a weighted mixture of such distributions, that similarly minimizes the KL-Divergence to the true posterior. By introducing two new ideas, namely, conditioning on groups of variables instead of single ones and using a parameter of the conditional random field potentials, that we identify to the temperature in the sense of statistical physics to select such groups, we can perform this minimization efficiently. Our extension of the clamping method proposed in previous works allows us to both produce a more descriptive approximation of the true posterior and, inspired by the diverse MAP paradigms, fit a mixture of Mean Field approximations. We demonstrate that this positively impacts real-world algorithms that initially relied on mean fields.