Improving GAN Equilibrium by Raising Spatial Awareness

TL;DR

This paper introduces a method to improve GAN training by enhancing the generator's spatial awareness through multi-level heatmaps and aligning it with the discriminator's attention, leading to more balanced adversarial training and better image synthesis.

Contribution

The paper proposes a novel approach to raise the generator's spatial awareness and align it with the discriminator's attention to improve GAN equilibrium and synthesis quality.

Findings

Enhanced GAN performance with closer equilibrium.

Improved image synthesis quality.

Facilitated interactive editing of generated images.

Abstract

The success of Generative Adversarial Networks (GANs) is largely built upon the adversarial training between a generator (G) and a discriminator (D). They are expected to reach a certain equilibrium where D cannot distinguish the generated images from the real ones. However, such an equilibrium is rarely achieved in practical GAN training, instead, D almost always surpasses G. We attribute one of its sources to the information asymmetry between D and G. We observe that D learns its own visual attention when determining whether an image is real or fake, but G has no explicit clue on which regions to focus on for a particular synthesis. To alleviate the issue of D dominating the competition in GANs, we aim to raise the spatial awareness of G. Randomly sampled multi-level heatmaps are encoded into the intermediate layers of G as an inductive bias. Thus G can purposefully improve the synthesis of certain image regions. We further propose to align the spatial awareness of G with the attention map induced from D. Through this way we effectively lessen the information gap between D and G. Extensive results show that our method pushes the two-player game in GANs closer to the equilibrium, leading to a better synthesis performance. As a byproduct, the introduced spatial awareness facilitates interactive editing over the output synthesis. Demo video and code are available at https://genforce.github.io/eqgan-sa/.