Discretization Drift in Two-Player Games

TL;DR

This paper investigates how discrete gradient updates in two-player games cause drift from continuous dynamics, affecting stability and performance, especially in GAN training, and proposes regularizers to mitigate harmful effects.

Contribution

It introduces a backward error analysis approach to understand discretization drift in two-player games and suggests regularizers to improve stability and performance.

Findings

Discretization drift can destabilize two-player game dynamics.

Modified continuous systems closely follow discrete updates.

Regularizers can cancel harmful drift and enhance GAN training.

Abstract

Gradient-based methods for two-player games produce rich dynamics that can solve challenging problems, yet can be difficult to stabilize and understand. Part of this complexity originates from the discrete update steps given by simultaneous or alternating gradient descent, which causes each player to drift away from the continuous gradient flow -- a phenomenon we call discretization drift. Using backward error analysis, we derive modified continuous dynamical systems that closely follow the discrete dynamics. These modified dynamics provide an insight into the notorious challenges associated with zero-sum games, including Generative Adversarial Networks. In particular, we identify distinct components of the discretization drift that can alter performance and in some cases destabilize the game. Finally, quantifying discretization drift allows us to identify regularizers that explicitly cancel harmful forms of drift or strengthen beneficial forms of drift, and thus improve performance of GAN training.