Leveraging Initial Hints for Free in Stochastic Linear Bandits

TL;DR

This paper introduces a new algorithm for stochastic linear bandits that effectively uses initial hints to improve regret bounds when hints are accurate, while maintaining optimal worst-case performance.

Contribution

The paper proposes a novel algorithm that leverages initial hints in stochastic linear bandits, achieving adaptive regret bounds and establishing tight tradeoffs with lower bounds.

Findings

Achieves $ ilde O(rac{1}{ ext{hint accuracy}} imes ext{poly}(d,rac{1}{ ext{hint accuracy}}))$ regret with accurate hints.

Maintains minimax-optimal $ ilde O(d oot T)$ regret regardless of hint quality.

Extends to multiple hints with $ ilde O(m^{2/3} oot T)$ regret.

Abstract

We study the setting of optimizing with bandit feedback with additional prior knowledge provided to the learner in the form of an initial hint of the optimal action. We present a novel algorithm for stochastic linear bandits that uses this hint to improve its regret to $\tilde O(\sqrt{T})$ when the hint is accurate, while maintaining a minimax-optimal $\tilde O(d\sqrt{T})$ regret independent of the quality of the hint. Furthermore, we provide a Pareto frontier of tight tradeoffs between best-case and worst-case regret, with matching lower bounds. Perhaps surprisingly, our work shows that leveraging a hint shows provable gains without sacrificing worst-case performance, implying that our algorithm adapts to the quality of the hint for free. We also provide an extension of our algorithm to the case of $m$ initial hints, showing that we can achieve a $\tilde O(m^{2/3}\sqrt{T})$ regret.