Optimistic and Adaptive Lagrangian Hedging

TL;DR

This paper introduces optimism and adaptive step sizes into Lagrangian hedging algorithms, enhancing their performance in online learning, offline optimization, and saddle point problems by leveraging problem structure for faster convergence.

Contribution

It extends Lagrangian hedging algorithms with optimism and adaptivity, providing new regret bounds and accelerating convergence in various online and non-adversarial settings.

Findings

Established a general regret bound for the new algorithms.

Derived path length regret bounds for smooth loss functions.

Provided optimistic regret bounds for $\

Abstract

In online learning an algorithm plays against an environment with losses possibly picked by an adversary at each round. The generality of this framework includes problems that are not adversarial, for example offline optimization, or saddle point problems (i.e. min max optimization). However, online algorithms are typically not designed to leverage additional structure present in non-adversarial problems. Recently, slight modifications to well-known online algorithms such as optimism and adaptive step sizes have been used in several domains to accelerate online learning -- recovering optimal rates in offline smooth optimization, and accelerating convergence to saddle points or social welfare in smooth games. In this work we introduce optimism and adaptive stepsizes to Lagrangian hedging, a class of online algorithms that includes regret-matching, and hedge (i.e. multiplicative weights). Our results include: a general general regret bound; a path length regret bound for a fixed smooth loss, applicable to an optimistic variant of regret-matching and regret-matching+; optimistic regret bounds for $\Phi$ regret, a framework that includes external, internal, and swap regret; and optimistic bounds for a family of algorithms that includes regret-matching+ as a special case.