Adversarial Generation of Continuous Images

TL;DR

This paper introduces novel INR-based image decoders and a state-of-the-art continuous image GAN, significantly improving generation quality and enabling properties like superresolution and interpolation.

Contribution

The paper proposes two new architectural techniques for INR-based decoders and develops a continuous image GAN that outperforms previous models on complex data.

Findings

Improved performance of continuous image GANs by several times.

Enables superresolution, interpolation, and boundary extrapolation with INR decoders.

Reduces the gap between continuous and pixel-based image generation.

Abstract

In most existing learning systems, images are typically viewed as 2D pixel arrays. However, in another paradigm gaining popularity, a 2D image is represented as an implicit neural representation (INR) - an MLP that predicts an RGB pixel value given its (x,y) coordinate. In this paper, we propose two novel architectural techniques for building INR-based image decoders: factorized multiplicative modulation and multi-scale INRs, and use them to build a state-of-the-art continuous image GAN. Previous attempts to adapt INRs for image generation were limited to MNIST-like datasets and do not scale to complex real-world data. Our proposed INR-GAN architecture improves the performance of continuous image generators by several times, greatly reducing the gap between continuous image GANs and pixel-based ones. Apart from that, we explore several exciting properties of the INR-based decoders, like out-of-the-box superresolution, meaningful image-space interpolation, accelerated inference of low-resolution images, an ability to extrapolate outside of image boundaries, and strong geometric prior. The project page is located at https://universome.github.io/inr-gan.