Constrained Online Convex Optimization with Memory and Predictions

TL;DR

This paper introduces algorithms for constrained online convex optimization with memory, achieving sublinear regret and constraint violation, and effectively utilizing predictions to improve performance in dynamic, constrained environments.

Contribution

The paper presents the first algorithms for COCO-M that handle time-varying constraints with and without predictions, extending online optimization to memory-dependent settings.

Findings

Achieves sublinear regret and constraint violation in COCO-M.

Develops adaptive penalty methods for no-prediction scenarios.

Designs optimistic algorithms that leverage predictions to enhance performance.

Abstract

We study Constrained Online Convex Optimization with Memory (COCO-M), where both the loss and the constraints depend on a finite window of past decisions made by the learner. This setting extends the previously studied unconstrained online optimization with memory framework and captures practical problems such as the control of constrained dynamical systems and scheduling with reconfiguration budgets. For this problem, we propose the first algorithms that achieve sublinear regret and sublinear cumulative constraint violation under time-varying constraints, both with and without predictions of future loss and constraint functions. Without predictions, we introduce an adaptive penalty approach that guarantees sublinear regret and constraint violation. When short-horizon and potentially unreliable predictions are available, we reinterpret the problem as online learning with delayed feedback and design an optimistic algorithm whose performance improves as prediction accuracy improves, while remaining robust when predictions are inaccurate. Our results bridge the gap between classical constrained online convex optimization and memory-dependent settings, and provide a versatile learning toolbox with diverse applications.