The Hidden Cost of Approximation in Online Mirror Descent

TL;DR

This paper investigates the impact of approximation errors in online mirror descent, revealing how regularizer smoothness influences robustness and providing bounds on regret under various regularizers and stochastic settings.

Contribution

It systematically analyzes inexact OMD, establishing bounds on regret related to regularizer smoothness and identifying conditions under which different regularizers are robust to errors.

Findings

Uniformly smooth regularizers have tight bounds on excess regret due to errors.

Negative entropy requires exponentially small errors to prevent linear regret.

Log-barrier and Tsallis regularizers are robust even with polynomial errors.

Abstract

Online mirror descent (OMD) is a fundamental algorithmic paradigm that underlies many algorithms in optimization, machine learning and sequential decision-making. The OMD iterates are defined as solutions to optimization subproblems which, oftentimes, can be solved only approximately, leading to an inexact version of the algorithm. Nonetheless, existing OMD analyses typically assume an idealized error free setting, thereby limiting our understanding of performance guarantees that should be expected in practice. In this work we initiate a systematic study into inexact OMD, and uncover an intricate relation between regularizer smoothness and robustness to approximation errors. When the regularizer is uniformly smooth, we establish a tight bound on the excess regret due to errors. Then, for barrier regularizers over the simplex and its subsets, we identify a sharp separation: negative entropy requires exponentially small errors to avoid linear regret, whereas log-barrier and Tsallis regularizers remain robust even when the errors are only polynomial. Finally, we show that when the losses are stochastic and the domain is the simplex, negative entropy regains robustness-but this property does not extend to all subsets, where exponentially small errors are again necessary to avoid suboptimal regret.