Stability Analysis Framework for Particle-based Distance GANs with Wasserstein Gradient Flow

TL;DR

This paper presents a stability analysis framework for particle-based distance GANs using Wasserstein gradient flow, identifying instability causes and proposing a stabilizing term to improve training stability.

Contribution

It introduces a novel stability analysis framework based on probability density dynamics and proposes a stabilizing term for the discriminator loss in particle-based distance GANs.

Findings

Discriminator training is often unstable due to the min-max formulation.

Adding a stabilizing term improves training stability.

Experimental results validate the effectiveness of the proposed method.

Abstract

In this paper, we investigate the training process of generative networks that use a type of probability density distance named particle-based distance as the objective function, e.g. MMD GAN, Cram\'er GAN, EIEG GAN. However, these GANs often suffer from the problem of unstable training. In this paper, we analyze the stability of the training process of these GANs from the perspective of probability density dynamics. In our framework, we regard the discriminator $D$ in these GANs as a feature transformation mapping that maps high dimensional data into a feature space, while the generator $G$ maps random variables to samples that resemble real data in terms of feature space. This perspective enables us to perform stability analysis for the training of GANs using the Wasserstein gradient flow of the probability density function. We find that the training process of the discriminator is usually unstable due to the formulation of $\min_G \max_D E(G, D)$ in GANs. To address this issue, we add a stabilizing term in the discriminator loss function. We conduct experiments to validate our stability analysis and stabilizing method.