The Minimal Search Space for Conditional Causal Bandits

TL;DR

This paper introduces a graphical method to identify the smallest set of nodes containing the optimal conditional intervention in causal bandits, significantly reducing search space and improving convergence speed.

Contribution

It provides a novel graphical characterization and an efficient algorithm for minimal search space identification in conditional causal bandits.

Findings

The algorithm correctly identifies the minimal node set.

Significant reduction in search space accelerates convergence.

Empirical results show improved performance in decision-making tasks.

Abstract

Causal knowledge can be used to support decision-making problems. This has been recognized in the causal bandits literature, where a causal (multi-armed) bandit is characterized by a causal graphical model and a target variable. The arms are then interventions on the causal model, and rewards are samples of the target variable. Causal bandits were originally studied with a focus on hard interventions. We focus instead on cases where the arms are conditional interventions, which more accurately model many real-world decision-making problems by allowing the value of the intervened variable to be chosen based on the observed values of other variables. This paper presents a graphical characterization of the minimal set of nodes guaranteed to contain the optimal conditional intervention, which maximizes the expected reward. We then propose an efficient algorithm with a time complexity of $O(|V| + |E|)$ to identify this minimal set of nodes. We prove that the graphical characterization and the proposed algorithm are correct. Finally, we empirically demonstrate that our algorithm significantly prunes the search space and substantially accelerates convergence rates when integrated into standard multi-armed bandit algorithms.