Recursive Reasoning in Minimax Games: A Level $k$ Gradient Play Method

TL;DR

This paper introduces the Level $k$ Gradient Play algorithm for stabilizing minimax game training, demonstrating convergence properties and practical advantages in GAN training with reduced computational resources.

Contribution

The paper proposes a novel recursive reasoning algorithm for minimax games that converges asymptotically and improves GAN training efficiency without complex heuristics.

Findings

Lv.$k$ GP converges to accurate strategy estimation as $k$ increases

Lv.$ ext{infinity}$ GP generalizes provably convergent dynamics

Achieves state-of-the-art GAN performance with fewer resources

Abstract

Despite the success of generative adversarial networks (GANs) in generating visually appealing images, they are notoriously challenging to train. In order to stabilize the learning dynamics in minimax games, we propose a novel recursive reasoning algorithm: Level $k$ Gradient Play (Lv.$k$ GP) algorithm. In contrast to many existing algorithms, our algorithm does not require sophisticated heuristics or curvature information. We show that as $k$ increases, Lv.$k$ GP converges asymptotically towards an accurate estimation of players' future strategy. Moreover, we justify that Lv.$\infty$ GP naturally generalizes a line of provably convergent game dynamics which rely on predictive updates. Furthermore, we provide its local convergence property in nonconvex-nonconcave zero-sum games and global convergence in bilinear and quadratic games. By combining Lv.$k$ GP with Adam optimizer, our algorithm shows a clear advantage in terms of performance and computational overhead compared to other methods. Using a single Nvidia RTX3090 GPU and 30 times fewer parameters than BigGAN on CIFAR-10, we achieve an FID of 10.17 for unconditional image generation within 30 hours, allowing GAN training on common computational resources to reach state-of-the-art performance.