Regret Bound by Variation for Online Convex Optimization

TL;DR

This paper extends variation-based regret bounds from online linear to general convex optimization, introduces new algorithms with such bounds, and explores bandit settings with optimal regret guarantees.

Contribution

It generalizes variation-based regret bounds to online convex optimization and proposes algorithms achieving these bounds, including in bandit scenarios.

Findings

Regret bounds are tied to the variation of cost functions.

New algorithms achieve variation-based regret bounds in convex and bandit settings.

Regret bounds are shown to be optimal when variation is independent of trial count.

Abstract

In citep{Hazan-2008-extract}, the authors showed that the regret of online linear optimization can be bounded by the total variation of the cost vectors. In this paper, we extend this result to general online convex optimization. We first analyze the limitations of the algorithm in \citep{Hazan-2008-extract} when applied it to online convex optimization. We then present two algorithms for online convex optimization whose regrets are bounded by the variation of cost functions. We finally consider the bandit setting, and present a randomized algorithm for online bandit convex optimization with a variation-based regret bound. We show that the regret bound for online bandit convex optimization is optimal when the variation of cost functions is independent of the number of trials.