Combinatorial Causal Bandits

TL;DR

This paper introduces algorithms for combinatorial causal bandits within binary generalized linear models, achieving near-optimal regret bounds and handling models with hidden variables without unrealistic assumptions.

Contribution

It presents the BGLM-OFU algorithm for Markovian models, extends to models with hidden variables using causal inference, and considers complex intervention spaces with minimal assumptions.

Findings

Achieves $O( ootT ext{log} T)$ regret for Markovian BGLMs.

Extends methods to linear models with hidden variables using do-calculus.

Handles complex causal models without assuming known joint distributions.

Abstract

In combinatorial causal bandits (CCB), the learning agent chooses at most $K$ variables in each round to intervene, collects feedback from the observed variables, with the goal of minimizing expected regret on the target variable $Y$. We study under the context of binary generalized linear models (BGLMs) with a succinct parametric representation of the causal models. We present the algorithm BGLM-OFU for Markovian BGLMs (i.e. no hidden variables) based on the maximum likelihood estimation method, and show that it achieves $O(\sqrt{T}\log T)$ regret, where $T$ is the time horizon. For the special case of linear models with hidden variables, we apply causal inference techniques such as the do-calculus to convert the original model into a Markovian model, and then show that our BGLM-OFU algorithm and another algorithm based on the linear regression both solve such linear models with hidden variables. Our novelty includes (a) considering the combinatorial intervention action space and the general causal models including ones with hidden variables, (b) integrating and adapting techniques from diverse studies such as generalized linear bandits and online influence maximization, and (c) avoiding unrealistic assumptions (such as knowing the joint distribution of the parents of $Y$ under all interventions) and regret factors exponential to causal graph size in prior studies.