Nearest Neighbour with Bandit Feedback

TL;DR

This paper introduces an efficient nearest neighbor algorithm for the adversarial contextual bandit problem, achieving low regret with polylogarithmic runtime and space, applicable to stochastic bandits and online classification.

Contribution

It adapts the nearest neighbor rule to the adversarial setting with a fast, scalable algorithm using adaptive search structures, providing generic regret bounds.

Findings

Algorithm achieves polylogarithmic per-trial runtime.

Provides regret bounds applicable to adversarial and stochastic settings.

Can be extended to online classification tasks.

Abstract

In this paper we adapt the nearest neighbour rule to the contextual bandit problem. Our algorithm handles the fully adversarial setting in which no assumptions at all are made about the data-generation process. When combined with a sufficiently fast data-structure for (perhaps approximate) adaptive nearest neighbour search, such as a navigating net, our algorithm is extremely efficient - having a per trial running time polylogarithmic in both the number of trials and actions, and taking only quasi-linear space. We give generic regret bounds for our algorithm and further analyse them when applied to the stochastic bandit problem in euclidean space. We note that our algorithm can also be applied to the online classification problem.