Spatial Frequency Bias in Convolutional Generative Adversarial Networks

TL;DR

This paper investigates the systemic bias of convolutional GANs against learning high spatial frequencies, revealing limitations in detail generation and proposing a method to manipulate this bias for improved control over generated image details.

Contribution

It uncovers a systemic bias in convolutional GANs against high spatial frequencies and introduces a simple method to manipulate this bias for better detail control.

Findings

GANs have a bias against high spatial frequencies.

This bias persists regardless of dataset frequency distribution.

A minimal-overhead method to manipulate the bias is proposed.

Abstract

As the success of Generative Adversarial Networks (GANs) on natural images quickly propels them into various real-life applications across different domains, it becomes more and more important to clearly understand their limitations. Specifically, understanding GANs' capability across the full spectrum of spatial frequencies, i.e. beyond the low-frequency dominant spectrum of natural images, is critical for assessing the reliability of GAN generated data in any detail-sensitive application (e.g. denoising, filling and super-resolution in medical and satellite images). In this paper, we show that the ability of convolutional GANs to learn a distribution is significantly affected by the spatial frequency of the underlying carrier signal, that is, GANs have a bias against learning high spatial frequencies. Crucially, we show that this bias is not merely a result of the scarcity of high frequencies in natural images, rather, it is a systemic bias hindering the learning of high frequencies regardless of their prominence in a dataset. Furthermore, we explain why large-scale GANs' ability to generate fine details on natural images does not exclude them from the adverse effects of this bias. Finally, we propose a method for manipulating this bias with minimal computational overhead. This method can be used to explicitly direct computational resources towards any specific spatial frequency of interest in a dataset, extending the flexibility of GANs.