Dueling Bandits with Dependent Arms
Bangrui Chen, Peter I. Frazier

TL;DR
This paper introduces the Comparing The Best (CTB) algorithm for dueling bandits with dependent arms, leveraging feature-based preferences to achieve constant regret and outperform benchmarks.
Contribution
It proposes a novel algorithm for dependent dueling bandits with theoretical guarantees and practical implementations for different arm quantities.
Findings
CTB achieves constant expected cumulative weak utility-based regret.
Numerical experiments show CTB outperforms benchmark algorithms.
The approach exploits dependence structure to improve learning efficiency.
Abstract
We study dueling bandits with weak utility-based regret when preferences over arms have a total order and carry observable feature vectors. The order is assumed to be determined by these feature vectors, an unknown preference vector, and a known utility function. This structure introduces dependence between preferences for pairs of arms, and allows learning about the preference over one pair of arms from the preference over another pair of arms. We propose an algorithm for this setting called Comparing The Best (CTB), which we show has constant expected cumulative weak utility-based regret. We provide a Bayesian interpretation for CTB, an implementation appropriate for a small number of arms, and an alternate implementation for many arms that can be used when the input parameters satisfy a decomposability condition. We demonstrate through numerical experiments that CTB with appropriate…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Bandit Algorithms Research · Auction Theory and Applications · Optimization and Search Problems
