A Practical Adaptive Subgame Perfect Gradient Method

TL;DR

This paper introduces ASPGM, an adaptive gradient method for smooth convex optimization that is linesearch-free, parameter-free, and provides strong theoretical guarantees, outperforming some existing methods.

Contribution

The paper proposes ASPGM, a novel adaptive gradient algorithm with subgame perfect optimality, offering theoretical guarantees and practical efficiency for convex optimization.

Findings

ASPGM is competitive with L-BFGS on various problems.

It is linesearch-free and parameter-free, simplifying implementation.

Provides certificates of solution quality and effective stopping criteria.

Abstract

We present a performant gradient method for smooth convex optimization, drawing inspiration from several recent advances in the field. Our algorithm, the Adaptive Subgame Perfect Gradient Method (ASPGM) is based on the notion of subgame perfection, attaining a dynamic strengthening of minimax optimality. At each iteration, ASPGM makes a momentum-type update, optimized dynamically based on a (limited) memory/bundle of past first-order information. ASPGM is linesearch-free, parameter-free, and adaptive due to its use of recently developed auto-conditioning, restarting, and preconditioning ideas. We show that ASPGM is competitive with state-of-the-art L-BFGS methods on a wide range of smooth convex problems. Unlike quasi-Newton methods, however, our core algorithm underlying ASPGM has strong, subgame perfect, non-asymptotic guarantees, providing certificates of solution quality, resulting in simple stopping criteria and restarting conditions.