An Optimistic Algorithm for Online Convex Optimization with Adversarial Constraints

TL;DR

This paper introduces an optimistic algorithm for online convex optimization with adversarial constraints, leveraging predictions to improve regret and constraint violation bounds, especially when predictions are accurate.

Contribution

The paper proposes a novel optimistic algorithm that improves bounds on regret and constraint violations in online convex optimization with adversarial constraints by exploiting prediction accuracy.

Findings

Improved bounds of $O(\sqrt{E_T(f)})$ for regret and $O(\sqrt{E_T(g^+)})$ for constraint violations.

Achieves better performance with accurate predictions, reducing regret and violations.

Extends to adversarial contextual bandits with risk constraints, providing optimistic bounds.

Abstract

We study Online Convex Optimization (OCO) with adversarial constraints, where an online algorithm must make sequential decisions to minimize both convex loss functions and cumulative constraint violations. We focus on a setting where the algorithm has access to predictions of the loss and constraint functions. Our results show that we can improve the current best bounds of $ O(\sqrt{T}) $ regret and $ \tilde{O}(\sqrt{T}) $ cumulative constraint violations to $ O(\sqrt{E_T(f)}) $ and $ \tilde{O}(\sqrt{E_T(g^+)}) $, respectively, where $ E_T(f) $ and $E_T(g^+)$ represent the cumulative prediction errors of the loss and constraint functions. In the worst case, where $E_T(f) = O(T) $ and $ E_T(g^+) = O(T) $ (assuming bounded gradients of the loss and constraint functions), our rates match the prior $ O(\sqrt{T}) $ results. However, when the loss and constraint predictions are accurate, our approach yields significantly smaller regret and cumulative constraint violations. Finally, we apply this to the setting of adversarial contextual bandits with sequential risk constraints, obtaining optimistic bounds $O (\sqrt{E_T(f)} T^{1/3})$ regret and $O(\sqrt{E_T(g^+)} T^{1/3})$ constraints violation, yielding better performance than existing results when prediction quality is sufficiently high.