Line-search and Adaptive Step Sizes for Nonconvex-strongly-concave Minimax Optimization

TL;DR

This paper introduces a new reformulation of nonconvex-strongly-concave minimax problems that enhances theoretical properties and develops a versatile, parameter-free line-search method compatible with GDA, improving convergence and numerical performance.

Contribution

It proposes a novel reformulation that preserves key optimality properties and introduces a flexible line-search framework compatible with GDA for better convergence.

Findings

Reformulation preserves stationarity and optimality properties.

The proposed method achieves global convergence rates.

Numerical experiments show superior performance over benchmarks.

Abstract

In this paper, we propose a novel reformulation of the smooth nonconvex-strongly-concave (NC-SC) minimax problems that casts the problem as a joint minimization. We show that our reformulation preserves not only first-order stationarity, but also global and local optimality, second-order stationarity, and the Kurdyka-{\L}ojasiewicz (KL) property, of the original NC-SC problem, which is substantially stronger than its nonsmooth counterpart in the literature. With these enhanced structures, we design a versatile parameter-free and nonmonotone line-search framework that does not require evaluating the inner maximization. Under mild conditions, global convergence rates can be obtained, and, with KL property, full sequence convergence with asymptotic rates is also established. In particular, we show our framework is compatible with the gradient descent-ascent (GDA) algorithm. By equipping GDA with Barzilai-Borwein (BB) step sizes and nonmonotone line-search, our method exhibits superior numerical performance against the compared benchmarks.