Penalised FTRL With Time-Varying Constraints

TL;DR

This paper extends the FTRL algorithm to handle time-varying constraints using adaptive penalization, achieving optimal regret and violation bounds in a broad class of constrained online learning problems.

Contribution

It introduces a penalized FTRL algorithm with adaptive penalization for time-varying constraints, providing necessary and sufficient conditions for optimal regret and violation bounds.

Findings

Achieves $O(\sqrt{t})$ regret and violation bounds.

Extends primal-dual algorithms to broader problem classes.

Provides necessary conditions for regret and violation guarantees.

Abstract

In this paper we extend the classical Follow-The-Regularized-Leader (FTRL) algorithm to encompass time-varying constraints, through adaptive penalization. We establish sufficient conditions for the proposed Penalized FTRL algorithm to achieve $O(\sqrt{t})$ regret and violation with respect to strong benchmark $\hat{X}^{max}_t$. Lacking prior knowledge of the constraints, this is probably the largest benchmark set that we can reasonably hope for. Our sufficient conditions are necessary in the sense that when they are violated there exist examples where $O(\sqrt{t})$ regret and violation is not achieved. Compared to the best existing primal-dual algorithms, Penalized FTRL substantially extends the class of problems for which $O(\sqrt{t})$ regret and violation performance is achievable.