Unsupervised Controllable Generation with Self-Training

TL;DR

This paper introduces an unsupervised self-training framework for GANs that learns semantically meaningful and disentangled latent codes, enabling controllable image generation without supervision, and demonstrates improved disentanglement and generation quality.

Contribution

The paper proposes a novel unsupervised self-training approach that learns a latent distribution for controllable GANs using tensor factorization, improving disentanglement and semantic control.

Findings

Better disentanglement than VAEs.

Discovering semantically meaningful latent codes.

Generating higher quality controllable images.

Abstract

Recent generative adversarial networks (GANs) are able to generate impressive photo-realistic images. However, controllable generation with GANs remains a challenging research problem. Achieving controllable generation requires semantically interpretable and disentangled factors of variation. It is challenging to achieve this goal using simple fixed distributions such as Gaussian distribution. Instead, we propose an unsupervised framework to learn a distribution of latent codes that control the generator through self-training. Self-training provides an iterative feedback in the GAN training, from the discriminator to the generator, and progressively improves the proposal of the latent codes as training proceeds. The latent codes are sampled from a latent variable model that is learned in the feature space of the discriminator. We consider a normalized independent component analysis model and learn its parameters through tensor factorization of the higher-order moments. Our framework exhibits better disentanglement compared to other variants such as the variational autoencoder, and is able to discover semantically meaningful latent codes without any supervision. We demonstrate empirically on both cars and faces datasets that each group of elements in the learned code controls a mode of variation with a semantic meaning, e.g. pose or background change. We also demonstrate with quantitative metrics that our method generates better results compared to other approaches.