A method for escaping limit cycles in training GANs

TL;DR

This paper introduces the predictive centripetal acceleration algorithm (PCAA) to mitigate limit cycle issues in GAN training, providing theoretical bounds and practical validation on various datasets.

Contribution

The paper develops PCAA with improved convergence bounds and combines it with Adam to create PCAA-Adam, a novel method for more stable GAN training.

Findings

PCAA improves convergence rates in bilinear games.

PCAA-Adam effectively reduces limit cycling in GAN training.

Experimental results demonstrate improved stability on multiple datasets.

Abstract

This paper mainly conducts further research to alleviate the issue of limit cycling behavior in training generative adversarial networks (GANs) through the proposed predictive centripetal acceleration algorithm (PCAA). Specifically, we first derive the upper and lower bounds on the last-iterate convergence rates of PCAA for the general bilinear game, with the upper bound notably improving upon previous results. Then, we combine PCAA with the adaptive moment estimation algorithm (Adam) to propose PCAA-Adam, a practical approach for training GANs. Finally, we validate the effectiveness of the proposed algorithm through experiments conducted on bilinear games, multivariate Gaussian distributions, and the CelebA dataset, respectively.