Causal Bandits: Online Decision-Making in Endogenous Settings

TL;DR

This paper introduces the $ ext{ extepsilon}$-BanditIV algorithm for online decision-making in endogenous linear bandit problems, using instrumental variables to correct bias, with proven regret bounds and demonstrated superior performance in simulations and real data.

Contribution

The paper proposes the $ ext{ extepsilon}$-BanditIV algorithm that addresses endogenous covariates in linear bandits using instrumental variables, with theoretical guarantees and practical validation.

Findings

$ ext{ extepsilon}$-BanditIV outperforms existing methods in endogenous settings.

The algorithm achieves $ ilde{ ext{O}}(k extsqrt{T})$ regret bounds.

Demonstrated effectiveness on real-time bidding data.

Abstract

The deployment of Multi-Armed Bandits (MAB) has become commonplace in many economic applications. However, regret guarantees for even state-of-the-art linear bandit algorithms (such as Optimism in the Face of Uncertainty Linear bandit (OFUL)) make strong exogeneity assumptions w.r.t. arm covariates. This assumption is very often violated in many economic contexts and using such algorithms can lead to sub-optimal decisions. Further, in social science analysis, it is also important to understand the asymptotic distribution of estimated parameters. To this end, in this paper, we consider the problem of online learning in linear stochastic contextual bandit problems with endogenous covariates. We propose an algorithm we term $\epsilon$-BanditIV, that uses instrumental variables to correct for this bias, and prove an $\tilde{\mathcal{O}}(k\sqrt{T})$ upper bound for the expected regret of the algorithm. Further, we demonstrate the asymptotic consistency and normality of the $\epsilon$-BanditIV estimator. We carry out extensive Monte Carlo simulations to demonstrate the performance of our algorithms compared to other methods. We show that $\epsilon$-BanditIV significantly outperforms other existing methods in endogenous settings. Finally, we use data from real-time bidding (RTB) system to demonstrate how $\epsilon$-BanditIV can be used to estimate the causal impact of advertising in such settings and compare its performance with other existing methods.