The Landscape of the Proximal Point Method for Nonconvex-Nonconcave Minimax Optimization

TL;DR

This paper analyzes the proximal point method for nonconvex-nonconcave minimax problems, revealing conditions for convergence and divergence based on the interaction strength between variables, with implications for machine learning applications.

Contribution

It introduces a novel analysis of the proximal point method using a generalized Moreau envelope, identifying three problem regions and their convergence behaviors.

Findings

Strong interaction leads to global linear convergence.

Weak interaction allows local convergence with proper initialization.

Intermediate interaction may cause divergence or limit cycles.

Abstract

Minimax optimization has become a central tool in machine learning with applications in robust optimization, reinforcement learning, GANs, etc. These applications are often nonconvex-nonconcave, but the existing theory is unable to identify and deal with the fundamental difficulties this poses. In this paper, we study the classic proximal point method (PPM) applied to nonconvex-nonconcave minimax problems. We find that a classic generalization of the Moreau envelope by Attouch and Wets provides key insights. Critically, we show this envelope not only smooths the objective but can convexify and concavify it based on the level of interaction present between the minimizing and maximizing variables. From this, we identify three distinct regions of nonconvex-nonconcave problems. When interaction is sufficiently strong, we derive global linear convergence guarantees. Conversely when the interaction is fairly weak, we derive local linear convergence guarantees with a proper initialization. Between these two settings, we show that PPM may diverge or converge to a limit cycle.