Banker Online Mirror Descent

TL;DR

Banker-OMD is a new framework that extends Online Mirror Descent to effectively handle delayed feedback in online learning, achieving near-optimal regret bounds across multiple bandit scenarios.

Contribution

It introduces Banker-OMD, a general methodology for delayed feedback online learning, with applications to multiple bandit problems and new regret guarantees.

Findings

Achieves $ ilde{O}( ext{poly}(n)( ext{sqrt}(T) + ext{sqrt}(D)))$ regret in delayed adversarial linear bandits.

Provides the first delayed adversarial linear bandit algorithm with near-optimal regret bounds.

Demonstrates robustness and versatility of Banker-OMD across different delayed-feedback online learning tasks.

Abstract

We propose Banker-OMD, a novel framework generalizing the classical Online Mirror Descent (OMD) technique in online learning algorithm design. Banker-OMD allows algorithms to robustly handle delayed feedback, and offers a general methodology for achieving $\tilde{O}(\sqrt{T} + \sqrt{D})$-style regret bounds in various delayed-feedback online learning tasks, where $T$ is the time horizon length and $D$ is the total feedback delay. We demonstrate the power of Banker-OMD with applications to three important bandit scenarios with delayed feedback, including delayed adversarial Multi-armed bandits (MAB), delayed adversarial linear bandits, and a novel delayed best-of-both-worlds MAB setting. Banker-OMD achieves nearly-optimal performance in all the three settings. In particular, it leads to the first delayed adversarial linear bandit algorithm achieving $\tilde{O}(\text{poly}(n)(\sqrt{T} + \sqrt{D}))$ regret.