Contextual Bandits Evolving Over Finite Time
Harsh Deshpande, Vishal Jain, Sharayu Moharir

TL;DR
This paper studies the challenges of contextual bandits with decaying externalities, analyzing existing policies and proposing a rejection-based method that ensures low regret regardless of reward structure.
Contribution
It introduces a rejection-based policy for evolving contextual bandits that guarantees low regret independent of the reward matrix structure.
Findings
Existing policies are biased towards the reward matrix.
The proposed rejection policy achieves low regret universally.
The approach handles externalities that decay over time.
Abstract
Contextual bandits have the same exploration-exploitation trade-off as standard multi-armed bandits. On adding positive externalities that decay with time, this problem becomes much more difficult as wrong decisions at the start are hard to recover from. We explore existing policies in this setting and highlight their biases towards the inherent reward matrix. We propose a rejection based policy that achieves a low regret irrespective of the structure of the reward probability matrix.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
