Penalty Gradient Normalization for Generative Adversarial Networks

TL;DR

This paper introduces penalty gradient normalization (PGN), a new method to stabilize GAN training by constraining discriminator gradient norms, leading to improved performance across multiple datasets.

Contribution

The paper proposes PGN, a novel normalization technique that enhances discriminator capacity and stabilizes GAN training, applicable to various architectures with minimal modifications.

Findings

PGN outperforms existing normalization methods in GAN training.

GANs with PGN achieve higher Frechet Inception Distance scores.

GANs with PGN attain better Inception Scores.

Abstract

In this paper, we propose a novel normalization method called penalty gradient normalization (PGN) to tackle the training instability of Generative Adversarial Networks (GANs) caused by the sharp gradient space. Unlike existing work such as gradient penalty and spectral normalization, the proposed PGN only imposes a penalty gradient norm constraint on the discriminator function, which increases the capacity of the discriminator. Moreover, the proposed penalty gradient normalization can be applied to different GAN architectures with little modification. Extensive experiments on three datasets show that GANs trained with penalty gradient normalization outperform existing methods in terms of both Frechet Inception and Distance and Inception Score.