SDE approximations of GANs training and its long-run behavior

TL;DR

This paper develops SDE-based models for GAN training, providing theoretical insights into its long-term behavior and stability through invariant measures, with precise error bounds for the approximations.

Contribution

It introduces a rigorous SDE approximation framework for GAN training and analyzes its long-run behavior, which was previously not well-understood.

Findings

SDE approximations accurately model GAN training dynamics

Invariant measures describe the long-term behavior of GANs

Provides analytical tools for studying GAN stability

Abstract

This paper analyzes the training process of GANs via stochastic differential equations (SDEs). It first establishes SDE approximations for the training of GANs under stochastic gradient algorithms, with precise error bound analysis. It then describes the long-run behavior of GANs training via the invariant measures of its SDE approximations under proper conditions. This work builds theoretical foundation for GANs training and provides analytical tools to study its evolution and stability.