On the Dynamic Regret of Following the Regularized Leader: Optimism with History Pruning

TL;DR

This paper improves the understanding of Follow the Regularized Leader (FTRL) in online convex optimization by introducing optimism and history pruning techniques to achieve better dynamic regret guarantees, balancing agility and stability.

Contribution

It presents a new analysis of FTRL that incorporates optimism and history pruning, enabling improved dynamic regret bounds and bridging lazy and agile update strategies.

Findings

FTRL can achieve known dynamic regret bounds with optimism and history pruning.

Pruning helps synchronize the algorithm's state with its iterates, improving regret.

The approach refines control over regret terms without cyclic dependence.

Abstract

We revisit the Follow the Regularized Leader (FTRL) framework for Online Convex Optimization (OCO) over compact sets, focusing on achieving dynamic regret guarantees. Prior work has highlighted the framework's limitations in dynamic environments due to its tendency to produce "lazy" iterates. However, building on insights showing FTRL's ability to produce "agile" iterates, we show that it can indeed recover known dynamic regret bounds through optimistic composition of future costs and careful linearization of past costs, which can lead to pruning some of them. This new analysis of FTRL against dynamic comparators yields a principled way to interpolate between lazy and agile updates and offers several benefits, including refined control over regret terms, optimism without cyclic dependence, and the application of minimal recursive regularization akin to AdaFTRL. More broadly, we show that it is not the "lazy" projection style of FTRL that hinders (optimistic) dynamic regret, but the decoupling of the algorithm's state (linearized history) from its iterates, allowing the state to grow arbitrarily. Instead, pruning synchronizes these two when necessary.