Adaptive Experiments Toward Learning Treatment Effect Heterogeneity

TL;DR

This paper introduces adaptive experimental designs that dynamically modify data collection to efficiently identify subgroups with significant treatment effects, improving personalized treatment and advertising strategies.

Contribution

It develops a unified framework for designing and analyzing response adaptive experiments specifically aimed at uncovering treatment effect heterogeneity.

Findings

Enhanced statistical efficiency in identifying treatment subgroups.

Theoretical justifications for adaptive design strategies.

Successful simulation results in e-commerce and clinical trial contexts.

Abstract

Understanding treatment effect heterogeneity has become an increasingly popular task in various fields, as it helps design personalized advertisements in e-commerce or targeted treatment in biomedical studies. However, most of the existing work in this research area focused on either analyzing observational data based on strong causal assumptions or conducting post hoc analyses of randomized controlled trial data, and there has been limited effort dedicated to the design of randomized experiments specifically for uncovering treatment effect heterogeneity. In the manuscript, we develop a framework for designing and analyzing response adaptive experiments toward better learning treatment effect heterogeneity. Concretely, we provide response adaptive experimental design frameworks that sequentially revise the data collection mechanism according to the accrued evidence during the experiment. Such design strategies allow for the identification of subgroups with the largest treatment effects with enhanced statistical efficiency. The proposed frameworks not only unify adaptive enrichment designs and response-adaptive randomization designs but also complement A/B test designs in e-commerce and randomized trial designs in clinical settings. We demonstrate the merit of our design with theoretical justifications and in simulation studies with synthetic e-commerce and clinical trial data.