Interpreting Spatially Infinite Generative Models

TL;DR

This paper provides a theoretical interpretation of spatially infinite generative models, connecting them to stochastic processes, and introduces an improved model called $ abla$-GAN that efficiently generates arbitrarily large images.

Contribution

It offers a theoretical framework for understanding infinite spatial generation and proposes an improved model, $ abla$-GAN, for more efficient training of such models.

Findings

$ abla$-GAN enables efficient training of infinite spatial generative models.

The models can generate images of arbitrary size, including world maps, panoramas, and textures.

Theoretical insights connect spatial generative models to stochastic processes.

Abstract

Traditional deep generative models of images and other spatial modalities can only generate fixed sized outputs. The generated images have exactly the same resolution as the training images, which is dictated by the number of layers in the underlying neural network. Recent work has shown, however, that feeding spatial noise vectors into a fully convolutional neural network enables both generation of arbitrary resolution output images as well as training on arbitrary resolution training images. While this work has provided impressive empirical results, little theoretical interpretation was provided to explain the underlying generative process. In this paper we provide a firm theoretical interpretation for infinite spatial generation, by drawing connections to spatial stochastic processes. We use the resulting intuition to improve upon existing spatially infinite generative models to enable more efficient training through a model that we call an infinite generative adversarial network, or $\infty$-GAN. Experiments on world map generation, panoramic images and texture synthesis verify the ability of $\infty$-GAN to efficiently generate images of arbitrary size.