Neural Variational Inference and Learning in Undirected Graphical Models

TL;DR

This paper introduces neural variational inference algorithms for undirected graphical models, enabling efficient learning, inference, and sampling by optimizing a neural network-based variational bound on the log-partition function.

Contribution

It presents a novel black-box variational inference method using neural networks to approximate the log-partition function in undirected models, unifying inference and learning.

Findings

Effective on multiple generative datasets

Allows tracking of the partition function during training

Speeds up sampling processes

Abstract

Many problems in machine learning are naturally expressed in the language of undirected graphical models. Here, we propose black-box learning and inference algorithms for undirected models that optimize a variational approximation to the log-likelihood of the model. Central to our approach is an upper bound on the log-partition function parametrized by a function q that we express as a flexible neural network. Our bound makes it possible to track the partition function during learning, to speed-up sampling, and to train a broad class of hybrid directed/undirected models via a unified variational inference framework. We empirically demonstrate the effectiveness of our method on several popular generative modeling datasets.