Hierarchical Causal Bandit
Ruiyang Song, Stefano Rini, Kuang Xu

TL;DR
This paper introduces the hierarchical causal bandit model, extending causal bandits to dependent variables by incorporating a contextual variable, and provides theoretical regret bounds for this new framework.
Contribution
It proposes a hierarchical framework for causal bandits with dependent variables and derives regret bounds, advancing understanding beyond independent-variable models.
Findings
Derived sharp regret bounds for the hierarchical causal bandit model.
Provided insights into algorithmic design for dependent causal bandits.
Extended the causal bandit theory to more realistic dependent-variable scenarios.
Abstract
Causal bandit is a nascent learning model where an agent sequentially experiments in a causal network of variables, in order to identify the reward-maximizing intervention. Despite the model's wide applicability, existing analytical results are largely restricted to a parallel bandit version where all variables are mutually independent. We introduce in this work the hierarchical causal bandit model as a viable path towards understanding general causal bandits with dependent variables. The core idea is to incorporate a contextual variable that captures the interaction among all variables with direct effects. Using this hierarchical framework, we derive sharp insights on algorithmic design in causal bandits with dependent arms and obtain nearly matching regret bounds in the case of a binary context.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Bandit Algorithms Research · Reinforcement Learning in Robotics · Machine Learning and Algorithms
