Synthesizing Programs for Images using Reinforced Adversarial Learning

TL;DR

This paper introduces SPIRAL, an unsupervised adversarial agent that generates high-level programs for images using reinforcement learning, effectively combining graphics engines with deep learning to interpret and generate images without supervision.

Contribution

The paper presents the first end-to-end, unsupervised adversarial approach for inverse graphics that successfully generates image programs using reinforcement learning and a discriminator-based reward.

Findings

Effective program generation for images using adversarial training.

Successful application to real-world datasets like MNIST, Omniglot, CelebA.

Demonstrates the importance of discriminator output as a reward signal.

Abstract

Advances in deep generative networks have led to impressive results in recent years. Nevertheless, such models can often waste their capacity on the minutiae of datasets, presumably due to weak inductive biases in their decoders. This is where graphics engines may come in handy since they abstract away low-level details and represent images as high-level programs. Current methods that combine deep learning and renderers are limited by hand-crafted likelihood or distance functions, a need for large amounts of supervision, or difficulties in scaling their inference algorithms to richer datasets. To mitigate these issues, we present SPIRAL, an adversarially trained agent that generates a program which is executed by a graphics engine to interpret and sample images. The goal of this agent is to fool a discriminator network that distinguishes between real and rendered data, trained with a distributed reinforcement learning setup without any supervision. A surprising finding is that using the discriminator's output as a reward signal is the key to allow the agent to make meaningful progress at matching the desired output rendering. To the best of our knowledge, this is the first demonstration of an end-to-end, unsupervised and adversarial inverse graphics agent on challenging real world (MNIST, Omniglot, CelebA) and synthetic 3D datasets.