Multi-armed Bandits with Missing Outcome
Ilia Mahrooghi, Mahshad Moradi, Sina Akbari, Negar Kiyavash

TL;DR
This paper addresses the challenge of missing outcomes in multi-armed bandit algorithms, proposing new methods to handle non-random missingness and improve regret bounds in practical scenarios.
Contribution
It introduces algorithms that explicitly account for missing data mechanisms, including MAR and MNAR, filling a significant gap in bandit literature.
Findings
Algorithms significantly reduce regret when handling missing outcomes.
Analytical results show improved regret bounds under different missingness models.
Simulation studies confirm practical effectiveness of the proposed methods.
Abstract
While significant progress has been made in designing algorithms that minimize regret in online decision-making, real-world scenarios often introduce additional complexities, perhaps the most challenging of which is missing outcomes. Overlooking this aspect or simply assuming random missingness invariably leads to biased estimates of the rewards and may result in linear regret. Despite the practical relevance of this challenge, no rigorous methodology currently exists for systematically handling missingness, especially when the missingness mechanism is not random. In this paper, we address this gap in the context of multi-armed bandits (MAB) with missing outcomes by analyzing the impact of different missingness mechanisms on achievable regret bounds. We introduce algorithms that account for missingness under both missing at random (MAR) and missing not at random (MNAR) models. Through…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Bandit Algorithms Research · COVID-19 epidemiological studies
