On Self Modulation for Generative Adversarial Networks

TL;DR

Self-modulation is a simple, label-free architectural modification for GANs that enhances performance across various settings by allowing generator feature maps to adapt based on input noise, reducing FID scores.

Contribution

We introduce self-modulation, a novel architectural technique for GANs that improves training stability and performance without additional parameters or labeled data.

Findings

Achieves 5-35% reduction in FID scores.

Improves performance in 86% of tested configurations.

Applicable across diverse datasets, architectures, and hyperparameters.

Abstract

Training Generative Adversarial Networks (GANs) is notoriously challenging. We propose and study an architectural modification, self-modulation, which improves GAN performance across different data sets, architectures, losses, regularizers, and hyperparameter settings. Intuitively, self-modulation allows the intermediate feature maps of a generator to change as a function of the input noise vector. While reminiscent of other conditioning techniques, it requires no labeled data. In a large-scale empirical study we observe a relative decrease of $5\%-35\%$ in FID. Furthermore, all else being equal, adding this modification to the generator leads to improved performance in $124/144$ ($86\%$) of the studied settings. Self-modulation is a simple architectural change that requires no additional parameter tuning, which suggests that it can be applied readily to any GAN.