Asymmetric regularization mechanism for GAN training with Variational Inequalities

TL;DR

This paper introduces an asymmetric regularization approach for GAN training formulated as a Nash equilibrium problem, ensuring convergence and stability through novel theoretical and empirical methods.

Contribution

It proposes a new asymmetric regularization mechanism based on Tikhonov regularization and a zero-centered gradient penalty, with theoretical convergence guarantees.

Findings

Ensures last-iterate linear convergence under certain conditions.

Empirical results demonstrate stabilization and convergence in GAN training.

Provides explicit Lipschitz and monotonicity constants for the regularized operator.

Abstract

We formulate the training of generative adversarial networks (GANs) as a Nash equilibrium seeking problem. To stabilize the training process and find a Nash equilibrium, we propose an asymmetric regularization mechanism based on the classic Tikhonov step and on a novel zero-centered gradient penalty. Under smoothness and a local identifiability condition induced by a Gauss-Newton Gramian, we obtain explicit Lipschitz and (strong)-monotonicity constants for the regularized operator. These constants ensure last-iterate linear convergence of a single-call Extrapolation-from-the-Past (EFTP) method. Empirical simulations on an academic example show that, even when strong monotonicity cannot be achieved, the asymmetric regularization is enough to converge to an equilibrium and stabilize the trajectory.