An Equivalence Between Static and Dynamic Regret Minimization

TL;DR

This paper establishes a fundamental equivalence between static and dynamic regret minimization in online convex optimization with linear losses, introduces a framework for various regret guarantees, and proposes an alternative variability measure with an effective algorithm.

Contribution

It reveals the equivalence between static and dynamic regret in linear settings and develops a unified framework for different regret bounds, including a new variability measure and algorithm.

Findings

Dynamic regret bounds are characterized by a trade-off frontier.

Adapting to squared path-length of comparator sequences is shown to be impossible.

An alternative variability measure enables effective dynamic regret guarantees.

Abstract

We study the problem of dynamic regret minimization in online convex optimization, in which the objective is to minimize the difference between the cumulative loss of an algorithm and that of an arbitrary sequence of comparators. While the literature on this topic is very rich, a unifying framework for the analysis and design of these algorithms is still missing. In this paper we show that for linear losses, dynamic regret minimization is equivalent to static regret minimization in an extended decision space. Using this simple observation, we show that there is a frontier of lower bounds trading off penalties due to the variance of the losses and penalties due to variability of the comparator sequence, and provide a framework for achieving any of the guarantees along this frontier. As a result, we also prove for the first time that adapting to the squared path-length of an arbitrary sequence of comparators to achieve regret $R_{T}(u_{1},\dots,u_{T})\le O(\sqrt{T\sum_{t} \|u_{t}-u_{t+1}\|^{2}})$ is impossible. However, using our framework we introduce an alternative notion of variability based on a locally-smoothed comparator sequence $\bar u_{1}, \dots, \bar u_{T}$, and provide an algorithm guaranteeing dynamic regret of the form $R_{T}(u_{1},\dots,u_{T})\le \tilde O(\sqrt{T\sum_{i}\|\bar u_{i}-\bar u_{i+1}\|^{2}})$, while still matching in the worst case the usual path-length dependencies up to polylogarithmic terms.