Doubly-Bounded Queue for Constrained Online Learning: Keeping Pace with Dynamics of Both Loss and Constraint

TL;DR

This paper introduces COLDQ, an online learning algorithm with a doubly-bounded queue that effectively manages dynamic regret and constraint violations without Slater condition, adapting to system dynamics.

Contribution

The paper proposes COLDQ, a novel algorithm with a virtual queue that bounds constraint violations and adapts to loss and constraint dynamics without Slater condition.

Findings

Achieves $O(T^{(1+V_x)/2})$ dynamic regret and $O(T^{V_g})$ constraint violation.

Matches $O(rac{1}{2})$ regret and $O(1)$ violation as dynamics diminish.

Outperforms state-of-the-art methods in simulations.

Abstract

We consider online convex optimization with time-varying constraints and conduct performance analysis using two stringent metrics: dynamic regret with respect to the online solution benchmark, and hard constraint violation that does not allow any compensated violation over time. We propose an efficient algorithm called Constrained Online Learning with Doubly-bounded Queue (COLDQ), which introduces a novel virtual queue that is both lower and upper bounded, allowing tight control of the constraint violation without the need for the Slater condition. We prove via a new Lyapunov drift analysis that COLDQ achieves $O(T^\frac{1+V_x}{2})$ dynamic regret and $O(T^{V_g})$ hard constraint violation, where $V_x$ and $V_g$ capture the dynamics of the loss and constraint functions. For the first time, the two bounds smoothly approach to the best-known $O(T^\frac{1}{2})$ regret and $O(1)$ violation, as the dynamics of the losses and constraints diminish. For strongly convex loss functions, COLDQ matches the best-known $O(\log{T})$ static regret while maintaining the $O(T^{V_g})$ hard constraint violation. We further introduce an expert-tracking variation of COLDQ, which achieves the same performance bounds without any prior knowledge of the system dynamics. Simulation results demonstrate that COLDQ outperforms the state-of-the-art approaches.