Constrained Online Convex Optimization with Polyak Feasibility Steps

TL;DR

This paper introduces a new online convex optimization method that guarantees constraint satisfaction at all times while maintaining optimal regret bounds, using Polyak feasibility steps to improve upon prior approaches.

Contribution

The paper proposes a novel approach employing Polyak feasibility steps to ensure anytime constraint satisfaction without sacrificing regret in online convex optimization.

Findings

Achieves $O( oot{T}{}$ regret with constraint satisfaction at all times.

Uses Polyak step-size for constraint updates, improving guarantees.

Validated through numerical experiments.

Abstract

In this work, we study online convex optimization with a fixed constraint function $g : \mathbb{R}^d \rightarrow \mathbb{R}$. Prior work on this problem has shown $O(\sqrt{T})$ regret and cumulative constraint satisfaction $\sum_{t=1}^{T} g(x_t) \leq 0$, while only accessing the constraint value and subgradient at the played actions $g(x_t), \partial g(x_t)$. Using the same constraint information, we show a stronger guarantee of anytime constraint satisfaction $g(x_t) \leq 0 \ \forall t \in [T]$, and matching $O(\sqrt{T})$ regret guarantees. These contributions are thanks to our approach of using Polyak feasibility steps to ensure constraint satisfaction, without sacrificing regret. Specifically, after each step of online gradient descent, our algorithm applies a subgradient descent step on the constraint function where the step-size is chosen according to the celebrated Polyak step-size. We further validate this approach with numerical experiments.