Any-resolution Training for High-resolution Image Synthesis

TL;DR

This paper introduces a continuous-scale training method for high-resolution image synthesis that leverages variable-size datasets to generate images at arbitrary resolutions with improved detail and global coherence.

Contribution

It proposes a novel training approach that conditions on continuous scales, enabling higher resolution generation without additional model complexity and better utilization of multi-resolution data.

Findings

Achieves higher FID scores and cleaner details than discrete multi-scale methods.

Enables arbitrary scale synthesis with coherent global layouts.

Demonstrates effectiveness across various natural image domains.

Abstract

Generative models operate at fixed resolution, even though natural images come in a variety of sizes. As high-resolution details are downsampled away and low-resolution images are discarded altogether, precious supervision is lost. We argue that every pixel matters and create datasets with variable-size images, collected at their native resolutions. To take advantage of varied-size data, we introduce continuous-scale training, a process that samples patches at random scales to train a new generator with variable output resolutions. First, conditioning the generator on a target scale allows us to generate higher resolution images than previously possible, without adding layers to the model. Second, by conditioning on continuous coordinates, we can sample patches that still obey a consistent global layout, which also allows for scalable training at higher resolutions. Controlled FFHQ experiments show that our method can take advantage of multi-resolution training data better than discrete multi-scale approaches, achieving better FID scores and cleaner high-frequency details. We also train on other natural image domains including churches, mountains, and birds, and demonstrate arbitrary scale synthesis with both coherent global layouts and realistic local details, going beyond 2K resolution in our experiments. Our project page is available at: https://chail.github.io/anyres-gan/.