Achieving Optimal Static and Dynamic Regret Simultaneously in Bandits with Deterministic Losses

TL;DR

This paper demonstrates the possibility of achieving optimal static and dynamic regret simultaneously in adversarial bandits with deterministic losses against an oblivious adversary, revealing a fundamental difference from adaptive adversaries.

Contribution

It extends the impossibility result to deterministic losses and introduces an algorithm that achieves optimal static and dynamic regret simultaneously against an oblivious adversary.

Findings

Optimal simultaneous static and dynamic regret is achievable against oblivious adversaries.

Impossibility results extend to deterministic losses, highlighting differences between adversary types.

The proposed algorithm leverages negative static regret and Blackwell approachability for joint regret control.

Abstract

In adversarial multi-armed bandits, two performance measures are commonly used: static regret, which compares the learner to the best fixed arm, and dynamic regret, which compares it to the best sequence of arms. While optimal algorithms are known for each measure individually, there is no known algorithm achieving optimal bounds for both simultaneously. Marinov and Zimmert [2021] first showed that such simultaneous optimality is impossible against an adaptive adversary. Our work takes a first step to demonstrate its possibility against an oblivious adversary when losses are deterministic. First, we extend the impossibility result of Marinov and Zimmert [2021] to the case of deterministic losses. Then, we present an algorithm achieving optimal static and dynamic regret simultaneously against an oblivious adversary. Together, they reveal a fundamental separation between adaptive and oblivious adversaries when multiple regret benchmarks are considered simultaneously. It also provides new insight into the long open problem of simultaneously achieving optimal regret against switching benchmarks of different numbers of switches. Our algorithm uses negative static regret to compensate for the exploration overhead incurred when controlling dynamic regret, and leverages Blackwell approachability to jointly control both regrets. This yields a new model selection procedure for bandits that may be of independent interest.