Tight Bounds for Online Convex Optimization with Adversarial Constraints

TL;DR

This paper proves that in constrained online convex optimization, it is possible to achieve both sublinear regret and constraint violation simultaneously, resolving a long-standing open problem.

Contribution

The paper introduces a novel online policy that attains $O( oot{T} ext{)}$ regret and constraint violation without restrictive assumptions, combining AdaGrad and Lyapunov optimization.

Findings

Achieves $O( oot{T} ext{)}$ regret and $ ilde{O}( oot{T} ext{)}$ constraint violation.

Provides a short and elegant analysis.

Answers a long-standing open question in constrained online convex optimization.

Abstract

A well-studied generalization of the standard online convex optimization (OCO) is constrained online convex optimization (COCO). In COCO, on every round, a convex cost function and a convex constraint function are revealed to the learner after the action for that round is chosen. The objective is to design an online policy that simultaneously achieves a small regret while ensuring small cumulative constraint violation (CCV) against an adaptive adversary. A long-standing open question in COCO is whether an online policy can simultaneously achieve $O(\sqrt{T})$ regret and $O(\sqrt{T})$ CCV without any restrictive assumptions. For the first time, we answer this in the affirmative and show that an online policy can simultaneously achieve $O(\sqrt{T})$ regret and $\tilde{O}(\sqrt{T})$ CCV. We establish this result by effectively combining the adaptive regret bound of the AdaGrad algorithm with Lyapunov optimization - a classic tool from control theory. Surprisingly, the analysis is short and elegant.