Causal Bandits without prior knowledge using separating sets

TL;DR

This paper introduces causal bandit algorithms that do not require prior causal graph knowledge, using separating sets and causal discovery methods to improve decision-making in complex, dependent reward environments.

Contribution

The paper proposes novel causal bandit algorithms leveraging separating sets and causal discovery, removing the need for prior causal graph knowledge.

Findings

Algorithms outperform existing methods in simulations

Effective on real-world protein signaling data

Unbiased estimator with bounded variance

Abstract

The Causal Bandit is a variant of the classic Bandit problem where an agent must identify the best action in a sequential decision-making process, where the reward distribution of the actions displays a non-trivial dependence structure that is governed by a causal model. Methods proposed for this problem thus far in the literature rely on exact prior knowledge of the full causal graph. We formulate new causal bandit algorithms that no longer necessarily rely on prior causal knowledge. Instead, they utilize an estimator based on separating sets, which we can find using simple conditional independence tests or causal discovery methods. We show that, given a true separating set, for discrete i.i.d. data, this estimator is unbiased, and has variance which is upper bounded by that of the sample mean. We develop algorithms based on Thompson Sampling and UCB for discrete and Gaussian models respectively and show increased performance on simulation data as well as on a bandit drawing from real-world protein signaling data.