Auxiliary Deep Generative Models

TL;DR

This paper introduces auxiliary variables into deep generative models to enhance variational approximation, leading to faster convergence and improved semi-supervised learning performance on multiple datasets.

Contribution

It proposes auxiliary variables that increase the expressiveness of variational distributions without changing the generative model structure.

Findings

Achieved state-of-the-art semi-supervised results on MNIST, SVHN, and NORB.

Models with auxiliary variables converge faster and perform better.

Introduced a model with two stochastic layers and skip connections.

Abstract

Deep generative models parameterized by neural networks have recently achieved state-of-the-art performance in unsupervised and semi-supervised learning. We extend deep generative models with auxiliary variables which improves the variational approximation. The auxiliary variables leave the generative model unchanged but make the variational distribution more expressive. Inspired by the structure of the auxiliary variable we also propose a model with two stochastic layers and skip connections. Our findings suggest that more expressive and properly specified deep generative models converge faster with better results. We show state-of-the-art performance within semi-supervised learning on MNIST, SVHN and NORB datasets.