Second Order Path Variationals in Non-Stationary Online Learning

TL;DR

This paper develops strongly adaptive algorithms for non-stationary online learning with exp-concave losses, achieving near-optimal dynamic regret bounds that adapt to second order path variations, especially for piecewise linear comparators.

Contribution

It introduces a novel analysis of second order path variationals and designs algorithms that attain optimal dynamic regret rates in non-stationary settings.

Findings

Achieves dynamic regret of  O(d^2 n^{1/5} C_n^{2/5}) for exp-concave losses.

Demonstrates the optimality of the regret rate up to dimension and logarithmic factors.

Extends analysis techniques to handle complex second order differences in comparator sequences.

Abstract

We consider the problem of universal dynamic regret minimization under exp-concave and smooth losses. We show that appropriately designed Strongly Adaptive algorithms achieve a dynamic regret of $\tilde O(d^2 n^{1/5} C_n^{2/5} \vee d^2)$, where $n$ is the time horizon and $C_n$ a path variational based on second order differences of the comparator sequence. Such a path variational naturally encodes comparator sequences that are piecewise linear -- a powerful family that tracks a variety of non-stationarity patterns in practice (Kim et al, 2009). The aforementioned dynamic regret rate is shown to be optimal modulo dimension dependencies and poly-logarithmic factors of $n$. Our proof techniques rely on analysing the KKT conditions of the offline oracle and requires several non-trivial generalizations of the ideas in Baby and Wang, 2021, where the latter work only leads to a slower dynamic regret rate of $\tilde O(d^{2.5}n^{1/3}C_n^{2/3} \vee d^{2.5})$ for the current problem.