F-Drop&Match: GANs with a Dead Zone in the High-Frequency Domain

TL;DR

This paper introduces F-Drop and F-Match, two techniques that improve GANs' ability to replicate high-frequency details in images by filtering and matching frequency components, leading to more realistic image generation.

Contribution

The paper proposes novel frequency-based training methods, F-Drop and F-Match, to enhance GANs' high-frequency component replication and overall image quality.

Findings

F-Drop prevents discriminator confusion by filtering high frequencies.

F-Match regularizes generator to match real image frequencies.

Combined methods improve GAN performance on multiple benchmarks.

Abstract

Generative adversarial networks built from deep convolutional neural networks (GANs) lack the ability to exactly replicate the high-frequency components of natural images. To alleviate this issue, we introduce two novel training techniques called frequency dropping (F-Drop) and frequency matching (F-Match). The key idea of F-Drop is to filter out unnecessary high-frequency components from the input images of the discriminators. This simple modification prevents the discriminators from being confused by perturbations of the high-frequency components. In addition, F-Drop makes the GANs focus on fitting in the low-frequency domain, in which there are the dominant components of natural images. F-Match minimizes the difference between real and fake images in the frequency domain for generating more realistic images. F-Match is implemented as a regularization term in the objective functions of the generators; it penalizes the batch mean error in the frequency domain. F-Match helps the generators to fit in the high-frequency domain filtered out by F-Drop to the real image. We experimentally demonstrate that the combination of F-Drop and F-Match improves the generative performance of GANs in both the frequency and spatial domain on multiple image benchmarks.