Online Sign Identification: Minimization of the Number of Errors in Thresholding Bandits
Reda Ouhamma, R\'emy Degenne, Pierre Gaillard, Vianney Perchet

TL;DR
This paper introduces a family of algorithms for the fixed budget thresholding bandit problem, demonstrating that adaptive methods outperform non-adaptive ones in minimizing errors, with theoretical guarantees and empirical evidence.
Contribution
It presents a new family of algorithms inspired by Frank-Wolfe, providing performance analysis and showing adaptive methods outperform non-adaptive oracles in this setting.
Findings
Adaptive algorithms outperform non-adaptive oracles empirically.
New explicit algorithms achieve losses close to non-adaptive oracle performance.
Surprising adaptive advantage explained through a toy problem.
Abstract
In the fixed budget thresholding bandit problem, an algorithm sequentially allocates a budgeted number of samples to different distributions. It then predicts whether the mean of each distribution is larger or lower than a given threshold. We introduce a large family of algorithms (containing most existing relevant ones), inspired by the Frank-Wolfe algorithm, and provide a thorough yet generic analysis of their performance. This allowed us to construct new explicit algorithms, for a broad class of problems, whose losses are within a small constant factor of the non-adaptive oracle ones. Quite interestingly, we observed that adaptive methods empirically greatly out-perform non-adaptive oracles, an uncommon behavior in standard online learning settings, such as regret minimization. We explain this surprising phenomenon on an insightful toy problem.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Taxonomy
TopicsAdvanced Bandit Algorithms Research · Mobile Crowdsensing and Crowdsourcing · Optimization and Search Problems
