
TL;DR
This paper introduces a random-effect model for multi-armed bandits, estimating arm means from data to improve regret bounds and outperform traditional methods like Thompson sampling without prior knowledge.
Contribution
It proposes a novel random-effect estimator and a UCB algorithm ReUCB that leverages this structure to enhance regret performance in bandit problems.
Findings
ReUCB achieves lower Bayes regret bounds compared to non-structured algorithms.
ReUCB outperforms Thompson sampling in experiments without prior distribution knowledge.
The random-effect estimator effectively captures the distribution of arm means.
Abstract
This paper studies regret minimization in a multi-armed bandit. It is well known that side information, such as the prior distribution of arm means in Thompson sampling, can improve the statistical efficiency of the bandit algorithm. While the prior is a blessing when correctly specified, it is a curse when misspecified. To address this issue, we introduce the assumption of a random-effect model to bandits. In this model, the mean arm rewards are drawn independently from an unknown distribution, which we estimate. We derive a random-effect estimator of the arm means, analyze its uncertainty, and design a UCB algorithm ReUCB that uses it. We analyze ReUCB and derive an upper bound on its -round Bayes regret, which improves upon not using the random-effect structure. Our experiments show that ReUCB can outperform Thompson sampling, without knowing the prior distribution of arm means.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Bandit Algorithms Research · Machine Learning and Algorithms · Data Stream Mining Techniques
