Semi-Anchored Multi-Step Gradient Descent Ascent Method for Structured Nonconvex-Nonconcave Composite Minimax Problems

TL;DR

This paper introduces SA-MGDA, a novel semi-anchored multi-step gradient descent ascent method that guarantees convergence to stationary points in structured nonconvex-nonconcave minimax problems, with practical applications demonstrated.

Contribution

It proposes a new semi-anchoring technique for MGDA, ensuring convergence in structured nonconvex-nonconcave minimax problems, extending the primal-dual hybrid gradient approach.

Findings

SA-MGDA finds stationary points in structured nonconvex-nonconcave minimax problems.

The method is built on a Bregman proximal point framework.

Numerical experiments demonstrate effectiveness in fair classification training.

Abstract

Minimax problems, such as generative adversarial network, adversarial training, and fair training, are widely solved by a multi-step gradient descent ascent (MGDA) method in practice. However, its convergence guarantee is limited. In this paper, inspired by the primal-dual hybrid gradient method, we propose a new semi-anchoring (SA) technique for the MGDA method. This makes the MGDA method find a stationary point of a structured nonconvex-nonconcave composite minimax problem; its saddle-subdifferential operator satisfies the weak Minty variational inequality condition. The resulting method, named SA-MGDA, is built upon a Bregman proximal point method. We further develop its backtracking line-search version, and its non-Euclidean version for smooth adaptable functions. Numerical experiments, including a fair classification training, are provided.