Newton and interior-point methods for (constrained) nonconvex-nonconcave minmax optimization with stability and instability guarantees

TL;DR

This paper introduces Newton and interior-point methods for nonconvex-nonconcave minmax problems, ensuring convergence to local minmax points and providing stability guarantees, with scalable computation demonstrated through numerical experiments.

Contribution

It proposes modified Newton-type algorithms with stability guarantees that converge only to local minmax solutions in nonconvex-nonconcave minmax optimization.

Findings

Algorithms only converge to local minmax points.

Modified Newton methods ensure stability and escape from non-minmax points.

Computation time scales linearly with the number of nonzero Hessian elements.

Abstract

We address the problem of finding a local solution to a nonconvex-nonconcave minmax optimization using Newton type methods, including interior-point ones. We modify the Hessian matrix of these methods such that, at each step, the modified Newton update direction can be seen as the solution to a quadratic program that locally approximates the minmax problem. Moreover, we show that by selecting the modification in an appropriate way, the only stable equilibrium points of the algorithm's iterations are local minmax points. As a consequence, the algorithm can only converge towards an equilibrium point if such point is a local minmax, and it will escape if the point is not a local minmax. Using numerical examples, we show that the computation time of our algorithm scales roughly linearly with the number of nonzero elements in the Hessian.