Optimality conditions for nonsmooth nonconvex-nonconcave min-max problems and generative adversarial networks

TL;DR

This paper develops new optimality conditions for complex nonsmooth, nonconvex-nonconcave min-max problems and applies these results to improve understanding of training generative adversarial networks.

Contribution

It introduces the first-order and second-order optimality conditions for local minimax points in nonsmooth, nonconvex-nonconcave problems, with applications to GANs.

Findings

Established conditions for the existence of global and local minimax points.

Derived optimality conditions using directional derivatives.

Applied theory to GAN training with illustrative examples.

Abstract

This paper considers a class of nonsmooth nonconvex-nonconcave min-max problems in machine learning and games. We first provide sufficient conditions for the existence of global minimax points and local minimax points. Next, we establish the first-order and second-order optimality conditions for local minimax points by using directional derivatives. These conditions reduce to smooth min-max problems with Fr{\'e}chet derivatives. We apply our theoretical results to generative adversarial networks (GANs) in which two neural networks contest with each other in a game. Examples are used to illustrate applications of the new theory for training GANs.