Bandits in Flux: Adversarial Constraints in Dynamic Environments

TL;DR

This paper introduces a primal-dual algorithm for adversarial multi-armed bandits with time-varying constraints, achieving sublinear regret and constraint violation, and demonstrating superior empirical performance.

Contribution

The paper proposes a novel primal-dual algorithm extending online mirror descent to handle adversarial bandits with dynamic constraints, providing theoretical guarantees and empirical validation.

Findings

Achieves sublinear dynamic regret and constraint violation.

Outperforms existing methods in empirical evaluations.

Provides theoretical guarantees for constrained adversarial bandit setting.

Abstract

We investigate the challenging problem of adversarial multi-armed bandits operating under time-varying constraints, a scenario motivated by numerous real-world applications. To address this complex setting, we propose a novel primal-dual algorithm that extends online mirror descent through the incorporation of suitable gradient estimators and effective constraint handling. We provide theoretical guarantees establishing sublinear dynamic regret and sublinear constraint violation for our proposed policy. Our algorithm achieves state-of-the-art performance in terms of both regret and constraint violation. Empirical evaluations demonstrate the superiority of our approach.