Partial Structure Discovery is Sufficient for No-regret Learning in Causal Bandits

TL;DR

This paper investigates causal bandit problems with unknown causal graphs, showing that discovering a partial causal structure suffices for no-regret learning, and introduces algorithms with sample complexity guarantees.

Contribution

It characterizes the necessary and sufficient components of the causal graph for optimal learning and proposes a two-stage algorithm combining causal discovery with bandit optimization.

Findings

Identifies minimal causal information needed for regret minimization.

Provides a sample complexity guarantee for causal graph learning.

Establishes a sublinear regret bound for the two-phase approach.

Abstract

Causal knowledge about the relationships among decision variables and a reward variable in a bandit setting can accelerate the learning of an optimal decision. Current works often assume the causal graph is known, which may not always be available a priori. Motivated by this challenge, we focus on the causal bandit problem in scenarios where the underlying causal graph is unknown and may include latent confounders. While intervention on the parents of the reward node is optimal in the absence of latent confounders, this is not necessarily the case in general. Instead, one must consider a set of possibly optimal arms/interventions, each being a special subset of the ancestors of the reward node, making causal discovery beyond the parents of the reward node essential. For regret minimization, we identify that discovering the full causal structure is unnecessary; however, no existing work provides the necessary and sufficient components of the causal graph. We formally characterize the set of necessary and sufficient latent confounders one needs to detect or learn to ensure that all possibly optimal arms are identified correctly. We also propose a randomized algorithm for learning the causal graph with a limited number of samples, providing a sample complexity guarantee for any desired confidence level. In the causal bandit setup, we propose a two-stage approach. In the first stage, we learn the induced subgraph on ancestors of the reward, along with a necessary and sufficient subset of latent confounders, to construct the set of possibly optimal arms. The regret incurred during this phase scales polynomially with respect to the number of nodes in the causal graph. The second phase involves the application of a standard bandit algorithm, such as the UCB algorithm. We also establish a regret bound for our two-phase approach, which is sublinear in the number of rounds.