PixelGAN Autoencoders

TL;DR

The PixelGAN autoencoder combines a PixelCNN generative model with a GAN-based recognition network to impose priors on latent codes, enabling unsupervised disentanglement and semi-supervised classification.

Contribution

It introduces a novel autoencoder architecture that integrates PixelCNN and GANs to control information decomposition and improve semi-supervised learning.

Findings

Different priors lead to various information decompositions.

Achieves unsupervised disentanglement of style and content.

Competitive semi-supervised classification results on benchmark datasets.

Abstract

In this paper, we describe the "PixelGAN autoencoder", a generative autoencoder in which the generative path is a convolutional autoregressive neural network on pixels (PixelCNN) that is conditioned on a latent code, and the recognition path uses a generative adversarial network (GAN) to impose a prior distribution on the latent code. We show that different priors result in different decompositions of information between the latent code and the autoregressive decoder. For example, by imposing a Gaussian distribution as the prior, we can achieve a global vs. local decomposition, or by imposing a categorical distribution as the prior, we can disentangle the style and content information of images in an unsupervised fashion. We further show how the PixelGAN autoencoder with a categorical prior can be directly used in semi-supervised settings and achieve competitive semi-supervised classification results on the MNIST, SVHN and NORB datasets.