Approximating exponential family models (not single distributions) with a two-network architecture

TL;DR

This paper introduces a two-network architecture for learning intractable exponential family models, enabling direct and generic posterior inference without needing to optimize for each specific distribution.

Contribution

The authors propose a novel two-network architecture and optimization method for learning entire exponential family models, not just single distributions, facilitating efficient inference.

Findings

Accurately learned exponential family models.

Enabled direct posterior inference from natural parameters.

Improved efficiency over traditional inference methods.

Abstract

Recently much attention has been paid to deep generative models, since they have been used to great success for variational inference, generation of complex data types, and more. In most all of these settings, the goal has been to find a particular member of that model family: optimized parameters index a distribution that is close (via a divergence or classification metric) to a target distribution. Much less attention, however, has been paid to the problem of learning a model itself. Here we introduce a two-network architecture and optimization procedure for learning intractable exponential family models (not a single distribution from those models). These exponential families are learned accurately, allowing operations like posterior inference to be executed directly and generically with an input choice of natural parameters, rather than performing inference via optimization for each particular distribution within that model.