Thompson sampling with the online bootstrap
Dean Eckles, Maurits Kaptein

TL;DR
This paper introduces bootstrap Thompson sampling (BTS), a scalable and robust heuristic for bandit problems that replaces the posterior with a bootstrap distribution, performing well even with model misspecification.
Contribution
The paper proposes BTS as a scalable, robust alternative to Thompson sampling, using online bootstrap to improve performance in large-scale and misspecified models.
Findings
BTS performs competitively with Thompson sampling in Bernoulli bandits.
BTS is more scalable than traditional Thompson sampling.
BTS shows robustness to error distribution misspecification.
Abstract
Thompson sampling provides a solution to bandit problems in which new observations are allocated to arms with the posterior probability that an arm is optimal. While sometimes easy to implement and asymptotically optimal, Thompson sampling can be computationally demanding in large scale bandit problems, and its performance is dependent on the model fit to the observed data. We introduce bootstrap Thompson sampling (BTS), a heuristic method for solving bandit problems which modifies Thompson sampling by replacing the posterior distribution used in Thompson sampling by a bootstrap distribution. We first explain BTS and show that the performance of BTS is competitive to Thompson sampling in the well-studied Bernoulli bandit case. Subsequently, we detail why BTS using the online bootstrap is more scalable than regular Thompson sampling, and we show through simulation that BTS is more robust…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Bandit Algorithms Research · Machine Learning and Algorithms · Data Stream Mining Techniques
