TSEC: a framework for online experimentation under experimental constraints
Simon Mak, Yuanshuo Zhou, Lavonne Hoang, C. F. Jeff Wu

TL;DR
This paper introduces TSEC, a Bayesian Thompson Sampling framework designed for online experimentation with constraints on the number of arms tested simultaneously, improving decision-making in large-scale, resource-limited settings.
Contribution
The paper presents a novel TSEC method that models correlations between arms using a Bayesian interaction model, enabling effective subset selection under experimental constraints.
Findings
TSEC outperforms industry benchmarks in website optimization.
TSEC achieves more consistent wealth accumulation in portfolio management.
Demonstrates effectiveness in simulated and real-world applications.
Abstract
Thompson sampling is a popular algorithm for solving multi-armed bandit problems, and has been applied in a wide range of applications, from website design to portfolio optimization. In such applications, however, the number of choices (or arms) can be large, and the data needed to make adaptive decisions require expensive experimentation. One is then faced with the constraint of experimenting on only a small subset of arms within each time period, which poses a problem for traditional Thompson sampling. We propose a new Thompson Sampling under Experimental Constraints (TSEC) method, which addresses this so-called "arm budget constraint". TSEC makes use of a Bayesian interaction model with effect hierarchy priors, to model correlations between rewards on different arms. This fitted model is then integrated within Thompson sampling, to jointly identify a good subset of arms…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Bandit Algorithms Research · Mobile Crowdsensing and Crowdsourcing · Machine Learning and Algorithms
