Fair Adaptive Experiments

TL;DR

This paper introduces a fair adaptive experiment method that improves data efficiency, ensures envy-free treatment allocation, and enhances participant welfare without relying on parametric outcome models.

Contribution

It proposes a novel, non-parametric adaptive treatment assignment strategy that guarantees fairness and asymptotic optimality, addressing fairness concerns in adaptive experiments.

Findings

Theoretical convergence rate of treatment effect estimates is established.

The algorithm asymptotically approaches the optimal allocation rule.

Simulation results demonstrate improved fairness and efficiency.

Abstract

Randomized experiments have been the gold standard for assessing the effectiveness of a treatment or policy. The classical complete randomization approach assigns treatments based on a prespecified probability and may lead to inefficient use of data. Adaptive experiments improve upon complete randomization by sequentially learning and updating treatment assignment probabilities. However, their application can also raise fairness and equity concerns, as assignment probabilities may vary drastically across groups of participants. Furthermore, when treatment is expected to be extremely beneficial to certain groups of participants, it is more appropriate to expose many of these participants to favorable treatment. In response to these challenges, we propose a fair adaptive experiment strategy that simultaneously enhances data use efficiency, achieves an envy-free treatment assignment guarantee, and improves the overall welfare of participants. An important feature of our proposed strategy is that we do not impose parametric modeling assumptions on the outcome variables, making it more versatile and applicable to a wider array of applications. Through our theoretical investigation, we characterize the convergence rate of the estimated treatment effects and the associated standard deviations at the group level and further prove that our adaptive treatment assignment algorithm, despite not having a closed-form expression, approaches the optimal allocation rule asymptotically. Our proof strategy takes into account the fact that the allocation decisions in our design depend on sequentially accumulated data, which poses a significant challenge in characterizing the properties and conducting statistical inference of our method. We further provide simulation evidence to showcase the performance of our fair adaptive experiment strategy.