A Homogeneous Second-Order Descent Ascent Algorithm for Nonconvex-Strongly Concave Minimax Problems

TL;DR

This paper presents a new second-order algorithm for nonconvex-strongly concave minimax problems that efficiently finds second-order stationary points, with practical variants suitable for large-scale applications.

Contribution

Introduction of the Homogeneous Second-order Descent Ascent (HSDA) algorithm and its inexact variant IHSDA, achieving optimal iteration complexity for nonconvex-strongly concave minimax problems.

Findings

HSDA guarantees descent directions with negative curvature.

HSDA finds an $ ilde{O}( ext{epsilon}^{-3/2})$-iteration complexity stationary point.

IHSDA maintains similar complexity with reduced computational cost.

Abstract

This paper introduces a novel Homogeneous Second-order Descent Ascent (HSDA) algorithm for nonconvex-strongly concave minimax optimization problems. At each iteration, HSDA uniquely computes a search direction by solving a homogenized eigenvalue subproblem built from the gradient and Hessian of the objective function. This formulation guarantees a descent direction with sufficient negative curvature even in near-positive-semidefinite Hessian regimes--a key feature that enhances escape from saddle points. We prove that HSDA finds an $\mathcal{O}(\varepsilon,\sqrt{\varepsilon})$-second-order stationary point within $\tilde{\mathcal{O}}(\varepsilon^{-3/2})$ iterations, matching the optimal $\varepsilon$-order iteration complexity among second-order methods for this problem class. To address large-scale applications, we further design an inexact variant (IHSDA) that preserves the single-loop structure while solving the subproblem approximately via a Lanczos procedure. With high probability, IHSDA achieves the same $\tilde{\mathcal{O}}(\varepsilon^{-3/2})$ iteration complexity and attains an $\mathcal{O}(\varepsilon, \sqrt{\varepsilon})$-second-order stationary point, with the total Hessian-vector product cost bounded by $\tilde{\mathcal{O}}(\varepsilon^{-7/4})$. Experiments on synthetic minimax problems and adversarial training tasks confirm the practical effectiveness and robustness of the proposed algorithms.