Adaptive and Optimal Second-order Optimistic Methods for Minimax Optimization

TL;DR

This paper introduces adaptive, line search-free second-order methods with optimal convergence rates for convex-concave minimax problems, combining optimistic updates with second-order information and adaptive step sizing.

Contribution

It presents novel adaptive second-order algorithms that do not require line search, using recursive step size rules based on gradient norms and prediction errors.

Findings

Achieves optimal convergence rates for minimax problems.

Requires only one linear system solve per iteration.

Demonstrates competitive practical performance against existing methods.

Abstract

We propose adaptive, line search-free second-order methods with optimal rate of convergence for solving convex-concave min-max problems. By means of an adaptive step size, our algorithms feature a simple update rule that requires solving only one linear system per iteration, eliminating the need for line search or backtracking mechanisms. Specifically, we base our algorithms on the optimistic method and appropriately combine it with second-order information. Moreover, distinct from common adaptive schemes, we define the step size recursively as a function of the gradient norm and the prediction error in the optimistic update. We first analyze a variant where the step size requires knowledge of the Lipschitz constant of the Hessian. Under the additional assumption of Lipschitz continuous gradients, we further design a parameter-free version by tracking the Hessian Lipschitz constant locally and ensuring the iterates remain bounded. We also evaluate the practical performance of our algorithm by comparing it to existing second-order algorithms for minimax optimization.