Minimax Optimization with Smooth Algorithmic Adversaries

TL;DR

This paper introduces a new minimax optimization algorithm that effectively handles nonconvex-nonconcave problems by playing against smooth algorithms, ensuring convergence and practical effectiveness in complex machine learning scenarios.

Contribution

It proposes a novel algorithm for minimax problems that plays against smooth algorithms rather than full maximization, guaranteeing convergence in challenging nonconvex-nonconcave settings.

Findings

Algorithm guarantees monotonic progress and avoids limit cycles.

Converges to stationary points in polynomial time.

Effective in practical machine learning applications like GANs.

Abstract

This paper considers minimax optimization $\min_x \max_y f(x, y)$ in the challenging setting where $f$ can be both nonconvex in $x$ and nonconcave in $y$. Though such optimization problems arise in many machine learning paradigms including training generative adversarial networks (GANs) and adversarially robust models, many fundamental issues remain in theory, such as the absence of efficiently computable optimality notions, and cyclic or diverging behavior of existing algorithms. Our framework sprouts from the practical consideration that under a computational budget, the max-player can not fully maximize $f(x,\cdot)$ since nonconcave maximization is NP-hard in general. So, we propose a new algorithm for the min-player to play against smooth algorithms deployed by the adversary (i.e., the max-player) instead of against full maximization. Our algorithm is guaranteed to make monotonic progress (thus having no limit cycles), and to find an appropriate "stationary point" in a polynomial number of iterations. Our framework covers practical settings where the smooth algorithms deployed by the adversary are multi-step stochastic gradient ascent, and its accelerated version. We further provide complementing experiments that confirm our theoretical findings and demonstrate the effectiveness of the proposed approach in practice.