Efficient Optimal Learning for Contextual Bandits

TL;DR

This paper introduces an efficient algorithm for contextual bandits that achieves optimal regret with significantly improved computational speed, enabling practical online learning with minimal delay impact.

Contribution

It presents the first computationally efficient algorithm with optimal regret for contextual bandits, using a cost-sensitive classification oracle and achieving polylogarithmic runtime.

Findings

Algorithm has polylogarithmic runtime in the number of classifiers.

Achieves optimal regret bounds in the online learning setting.

Handles feedback delay additively rather than multiplicatively.

Abstract

We address the problem of learning in an online setting where the learner repeatedly observes features, selects among a set of actions, and receives reward for the action taken. We provide the first efficient algorithm with an optimal regret. Our algorithm uses a cost sensitive classification learner as an oracle and has a running time $\mathrm{polylog}(N)$, where $N$ is the number of classification rules among which the oracle might choose. This is exponentially faster than all previous algorithms that achieve optimal regret in this setting. Our formulation also enables us to create an algorithm with regret that is additive rather than multiplicative in feedback delay as in all previous work.