Nested bandits
Matthieu Martin, Panayotis Mertikopoulos, Thibaud Rahier and, Houssam Zenati

TL;DR
This paper introduces the nested exponential weights (NEW) algorithm for hierarchical bandit problems, effectively reducing regret by layered exploration in environments with many similar alternatives.
Contribution
It proposes a novel layered exploration algorithm tailored for nested bandits, improving regret bounds in settings with hierarchical similarities.
Findings
NEW algorithm achieves tighter regret bounds
Efficiently handles large sets with hierarchical similarities
Prevents excessive exploration of irrelevant options
Abstract
In many online decision processes, the optimizing agent is called to choose between large numbers of alternatives with many inherent similarities; in turn, these similarities imply closely correlated losses that may confound standard discrete choice models and bandit algorithms. We study this question in the context of nested bandits, a class of adversarial multi-armed bandit problems where the learner seeks to minimize their regret in the presence of a large number of distinct alternatives with a hierarchy of embedded (non-combinatorial) similarities. In this setting, optimal algorithms based on the exponential weights blueprint (like Hedge, EXP3, and their variants) may incur significant regret because they tend to spend excessive amounts of time exploring irrelevant alternatives with similar, suboptimal costs. To account for this, we propose a nested exponential weights (NEW)…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Bandit Algorithms Research · Machine Learning and Algorithms · Data Stream Mining Techniques
