A Single-Loop Stochastic Gradient Algorithm for Minimax Optimization with Nonlinear Coupled Constraints

TL;DR

This paper introduces SPACO, a single-loop stochastic gradient algorithm designed for nonconvex-concave minimax problems with nonlinear coupled constraints, offering convergence guarantees and practical effectiveness.

Contribution

It presents a novel penalty-based smoothing framework and an algorithm with theoretical convergence analysis for complex constrained minimax optimization.

Findings

The method converges under certain conditions.

Non-asymptotic complexity bounds are established.

Experimental results show practical efficiency.

Abstract

In this paper, we propose a single-loop stochastic gradient algorithm for solving stochastic nonconvex-concave minimax optimization with nonlinear convex coupled constraints (MCC). The proposed method, SPACO (Stochastic Penalty-based Algorithm for minimax optimization with COupled constraints), is built upon a penalty-based smooth approximation framework for MCC. This framework integrates a quadratic penalty scheme with regularization to yield a continuously differentiable approximation of the MCC problem. We provide theoretical convergence guarantees for this smoothing framework. Furthermore, we establish non-asymptotic complexity bounds and provide an asymptotic analysis characterizing the stationarity of accumulation points for the iterates generated by SPACO. Experimental results on synthetic examples and practical machine learning tasks demonstrate the effectiveness and efficiency of the proposed method.