Online mirror descent and dual averaging: keeping pace in the dynamic case

TL;DR

This paper compares online mirror descent and dual averaging algorithms, introduces a stabilization technique for OMD, and demonstrates that both methods can achieve similar performance guarantees under dynamic learning rates.

Contribution

The authors propose a stabilization method for OMD, providing a unified analysis that shows comparable guarantees to DA even with dynamic learning rates.

Findings

Stabilized OMD matches DA in performance guarantees.

Both algorithms achieve similar regret bounds under dynamic learning rates.

Conditions identified where stabilized OMD and DA produce identical iterates.

Abstract

Online mirror descent (OMD) and dual averaging (DA) -- two fundamental algorithms for online convex optimization -- are known to have very similar (and sometimes identical) performance guarantees when used with a fixed learning rate. Under dynamic learning rates, however, OMD is provably inferior to DA and suffers a linear regret, even in common settings such as prediction with expert advice. We modify the OMD algorithm through a simple technique that we call stabilization. We give essentially the same abstract regret bound for OMD with stabilization and for DA by modifying the classical OMD convergence analysis in a careful and modular way that allows for straightforward and flexible proofs. Simple corollaries of these bounds show that OMD with stabilization and DA enjoy the same performance guarantees in many applications -- even under dynamic learning rates. We also shed light on the similarities between OMD and DA and show simple conditions under which stabilized-OMD and DA generate the same iterates.