Smoothing Meets Perturbation: Unified and Tight Analysis for Nonconvex-Concave Minimax Optimization

TL;DR

This paper analyzes nonconvex-concave minimax optimization, comparing smoothing and dual perturbation techniques, and introduces a new method that combines both to improve convergence guarantees.

Contribution

It provides a tight characterization of iteration complexities for smoothing and perturbation, and proposes a unified method that leverages both techniques for better convergence.

Findings

Smoothing accelerates convergence to game and optimization stationarity.

Dual perturbation improves convergence only for game stationarity.

The proposed Perturbed Smoothed GDA outperforms existing methods and converges asymptotically to 0-GS.

Abstract

This paper studies smooth nonconvex-concave minimax optimization and two acceleration mechanisms for single-loop first-order methods: dual perturbation and smoothing. Although both techniques improve convergence guarantees, their relative advantages remain unclear due to the distinction between game stationarity (GS) and optimization stationarity (OS). We provide a tight characterization of their iteration complexities under both notions. We show that smoothing accelerates convergence to both GS and OS, whereas dual perturbation improves the rate only for GS and does not accelerate OS. Matching lower bounds based on hard instances establish the tightness of these rates. Motivated by this separation, we propose Perturbed Smoothed GDA, a single-loop method combining both techniques. It improves the complexity for GS over existing single-loop methods while preserving the state-of-the-art rate for OS, and further admits asymptotic convergence to 0-GS, which is not available for vanilla Smoothed GDA.