Optimizing the Latent Space of Generative Networks

TL;DR

This paper introduces Generative Latent Optimization (GLO), a new method for training deep convolutional generators that achieves many benefits of GANs without using adversarial training, simplifying the process.

Contribution

GLO provides a non-adversarial training framework for deep generative models, disentangling the effects of adversarial training from network architecture.

Findings

GLO synthesizes visually appealing images.

GLO enables meaningful interpolation between samples.

GLO supports linear arithmetic in latent space.

Abstract

Generative Adversarial Networks (GANs) have achieved remarkable results in the task of generating realistic natural images. In most successful applications, GAN models share two common aspects: solving a challenging saddle point optimization problem, interpreted as an adversarial game between a generator and a discriminator functions; and parameterizing the generator and the discriminator as deep convolutional neural networks. The goal of this paper is to disentangle the contribution of these two factors to the success of GANs. In particular, we introduce Generative Latent Optimization (GLO), a framework to train deep convolutional generators using simple reconstruction losses. Throughout a variety of experiments, we show that GLO enjoys many of the desirable properties of GANs: synthesizing visually-appealing samples, interpolating meaningfully between samples, and performing linear arithmetic with noise vectors; all of this without the adversarial optimization scheme.