Adversarially-learned Inference via an Ensemble of Discrete Undirected Graphical Models

TL;DR

This paper introduces an adversarial training framework that creates an ensemble of graphical models, enabling better generalization to unseen inference tasks and faster data sampling compared to traditional models.

Contribution

It proposes a novel inference-agnostic adversarial training method to produce an ensemble of graphical models that generalize well to new inference tasks.

Findings

AGMs perform comparably on trained inference tasks

AGMs generalize better to unseen inference tasks

AGMs enable fast data sampling

Abstract

Undirected graphical models are compact representations of joint probability distributions over random variables. To solve inference tasks of interest, graphical models of arbitrary topology can be trained using empirical risk minimization. However, to solve inference tasks that were not seen during training, these models (EGMs) often need to be re-trained. Instead, we propose an inference-agnostic adversarial training framework which produces an infinitely-large ensemble of graphical models (AGMs). The ensemble is optimized to generate data within the GAN framework, and inference is performed using a finite subset of these models. AGMs perform comparably with EGMs on inference tasks that the latter were specifically optimized for. Most importantly, AGMs show significantly better generalization to unseen inference tasks compared to EGMs, as well as deep neural architectures like GibbsNet and VAEAC which allow arbitrary conditioning. Finally, AGMs allow fast data sampling, competitive with Gibbs sampling from EGMs.