On Dynamic Regret and Constraint Violations in Constrained Online Convex Optimization

TL;DR

This paper introduces an algorithm for constrained online convex optimization that effectively balances minimizing dynamic regret and constraint violations, with bounds tied to the path-length of optimal actions.

Contribution

The paper presents a novel projected gradient descent algorithm with optimal bounds for dynamic regret and constraint violations in constrained OCO.

Findings

Both dynamic regret and constraint violation are bounded by the path-length.

The bounds achieved are proven to be optimal.

The algorithm operates with only one gradient evaluation per iteration.

Abstract

A constrained version of the online convex optimization (OCO) problem is considered. With slotted time, for each slot, first an action is chosen. Subsequently the loss function and the constraint violation penalty evaluated at the chosen action point is revealed. For each slot, both the loss function as well as the function defining the constraint set is assumed to be smooth and strongly convex. In addition, once an action is chosen, local information about a feasible set within a small neighborhood of the current action is also revealed. An algorithm is allowed to compute at most one gradient at its point of choice given the described feedback to choose the next action. The goal of an algorithm is to simultaneously minimize the dynamic regret (loss incurred compared to the oracle's loss) and the constraint violation penalty (penalty accrued compared to the oracle's penalty). We propose an algorithm that follows projected gradient descent over a suitably chosen set around the current action. We show that both the dynamic regret and the constraint violation is order-wise bounded by the {\it path-length}, the sum of the distances between the consecutive optimal actions. Moreover, we show that the derived bounds are the best possible.