Learning Convex Inference of Marginals

TL;DR

This paper introduces a novel method for learning graphical models by directly optimizing the accuracy of marginal predictions through convex inference, improving robustness when inference or models are approximate.

Contribution

It proposes a new approach that directly minimizes empirical risk based on the performance of convex inference for marginal prediction, addressing limitations of traditional maximum likelihood methods.

Findings

Effective in handling approximate inference and models

Improves marginal prediction accuracy

Provides a convex optimization framework

Abstract

Graphical models trained using maximum likelihood are a common tool for probabilistic inference of marginal distributions. However, this approach suffers difficulties when either the inference process or the model is approximate. In this paper, the inference process is first defined to be the minimization of a convex function, inspired by free energy approximations. Learning is then done directly in terms of the performance of the inference process at univariate marginal prediction. The main novelty is that this is a direct minimization of emperical risk, where the risk measures the accuracy of predicted marginals.