Projection-Free Algorithms for Minimax Problems

TL;DR

This paper introduces a unified dual dynamic smoothing framework for projection-free minimax optimization, enabling efficient algorithms with convergence guarantees in various settings.

Contribution

It develops a novel framework that bridges projection-based and projection-free methods, providing the first unified approach with theoretical convergence results.

Findings

Established convergence results for nonconvex-concave and nonconvex-strongly concave problems.

Designed three variants using linear minimization oracles for different variables.

Achieved anytime convergence guarantees without fixed iteration limits.

Abstract

This paper addresses constrained smooth saddle-point problems in settings where projection onto the feasible sets is computationally expensive. We bridge the gap between projection-based and projection-free optimization by introducing a unified dual dynamic smoothing framework that enables the design of efficient single-loop algorithms. Within this framework, we establish convergence results for nonconvex-concave and nonconvex-strongly concave settings. Furthermore, we show that this framework is naturally applicable to convex-concave problems. We propose and analyze three algorithmic variants based on the application of a linear minimization oracle over the minimization variable, the maximization variable, or both. Notably, our analysis yields anytime convergence guarantees without requiring a pre-specified iteration horizon. These results significantly narrow the performance gap between projection-free and projection-based methods for minimax optimization.