Multi-Armed Bandits in Metric Spaces
Robert Kleinberg, Aleksandrs Slivkins, Eli Upfal

TL;DR
This paper studies multi-armed bandit problems where strategies are points in a metric space and payoffs are Lipschitz continuous, providing a complete solution with bounds and near-optimal algorithms.
Contribution
It introduces the Lipschitz MAB problem, defines an isometry invariant for performance bounds, and offers algorithms that approach these bounds in general and benign cases.
Findings
Established a lower bound on algorithm performance based on isometry invariants.
Developed algorithms that nearly achieve the performance bounds.
Achieved improved results for benign payoff functions.
Abstract
In a multi-armed bandit problem, an online algorithm chooses from a set of strategies in a sequence of trials so as to maximize the total payoff of the chosen strategies. While the performance of bandit algorithms with a small finite strategy set is quite well understood, bandit problems with large strategy sets are still a topic of very active investigation, motivated by practical applications such as online auctions and web advertisement. The goal of such research is to identify broad and natural classes of strategy sets and payoff functions which enable the design of efficient solutions. In this work we study a very general setting for the multi-armed bandit problem in which the strategies form a metric space, and the payoff function satisfies a Lipschitz condition with respect to the metric. We refer to this problem as the "Lipschitz MAB problem". We present a complete solution for…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Bandit Algorithms Research · Reinforcement Learning in Robotics · Smart Grid Energy Management
