Toward Spatially Unbiased Generative Models

TL;DR

This paper identifies spatial bias in image generators caused by implicit positional encoding and proposes injecting explicit positional encoding to create spatially unbiased models, improving robustness across various tasks.

Contribution

It introduces a method to inject explicit positional encoding into generators, reducing spatial bias and enhancing their versatility in multiple image generation tasks.

Findings

Reduced spatial bias in generators

Improved performance in multi-scale and arbitrary size generation

Applicable to diffusion models as well

Abstract

Recent image generation models show remarkable generation performance. However, they mirror strong location preference in datasets, which we call spatial bias. Therefore, generators render poor samples at unseen locations and scales. We argue that the generators rely on their implicit positional encoding to render spatial content. From our observations, the generator's implicit positional encoding is translation-variant, making the generator spatially biased. To address this issue, we propose injecting explicit positional encoding at each scale of the generator. By learning the spatially unbiased generator, we facilitate the robust use of generators in multiple tasks, such as GAN inversion, multi-scale generation, generation of arbitrary sizes and aspect ratios. Furthermore, we show that our method can also be applied to denoising diffusion probabilistic models.