TL;DR
This paper introduces probing strategies in online learning and multi-armed bandit problems, significantly improving regret bounds by using limited hints before making decisions, and demonstrates the power of such advice in various models.
Contribution
It develops algorithms with exponentially improved regret bounds using probing, including constant regret with few probes, and extends to imperfect and correlated reward settings.
Findings
Probing with 2 choices yields time-independent regret in online convex optimization.
Using 3 probes in stochastic MAB achieves parameter-independent constant regret.
Limited probing can outperform full feedback in regret bounds.
Abstract
We consider the classic online learning and stochastic multi-armed bandit (MAB) problems, when at each step, the online policy can probe and find out which of a small number () of choices has better reward (or loss) before making its choice. In this model, we derive algorithms whose regret bounds have exponentially better dependence on the time horizon compared to the classic regret bounds. In particular, we show that probing with suffices to achieve time-independent regret bounds for online linear and convex optimization. The same number of probes improve the regret bound of stochastic MAB with independent arms from to , where is the number of arms and is the horizon length. For stochastic MAB, we also consider a stronger model where a probe reveals the reward values of the probed arms, and show that in this case, probes suffice to…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Online Learning and Bandits with Queried Hints· youtube
