Adaptive Experimentation When You Can't Experiment

TL;DR

This paper develops an adaptive experimental design method for confounded pure exploration in linear bandits, addressing challenges where direct experimentation is infeasible and noise is confounded, with theoretical guarantees and empirical validation.

Contribution

Introduces the CPET-LB problem and a novel elimination algorithm with confidence intervals for confounded linear bandits in encouragement designs.

Findings

Algorithm achieves near-minimax optimal sample complexity.

Method effectively handles confounded noise in pure exploration.

Experiments validate the approach's practical efficacy.

Abstract

This paper introduces the \emph{confounded pure exploration transductive linear bandit} (\texttt{CPET-LB}) problem. As a motivating example, often online services cannot directly assign users to specific control or treatment experiences either for business or practical reasons. In these settings, naively comparing treatment and control groups that may result from self-selection can lead to biased estimates of underlying treatment effects. Instead, online services can employ a properly randomized encouragement that incentivizes users toward a specific treatment. Our methodology provides online services with an adaptive experimental design approach for learning the best-performing treatment for such \textit{encouragement designs}. We consider a more general underlying model captured by a linear structural equation and formulate pure exploration linear bandits in this setting. Though pure exploration has been extensively studied in standard adaptive experimental design settings, we believe this is the first work considering a setting where noise is confounded. Elimination-style algorithms using experimental design methods in combination with a novel finite-time confidence interval on an instrumental variable style estimator are presented with sample complexity upper bounds nearly matching a minimax lower bound. Finally, experiments are conducted that demonstrate the efficacy of our approach.