Interactive identification of individuals with positive treatment effect while controlling false discoveries

TL;DR

This paper introduces an interactive algorithm for identifying individuals with positive treatment effects in randomized experiments, controlling false discoveries, and extending to observational data with robust guarantees.

Contribution

It presents a novel, interactive method for individual treatment effect identification with finite-sample FDR control, adaptable to various settings and extensions.

Findings

Valid FDR control demonstrated in simulations

High power in identifying positive effects

Method extended to observational data with robustness

Abstract

Out of the participants in a randomized experiment with anticipated heterogeneous treatment effects, is it possible to identify which subjects have a positive treatment effect? While subgroup analysis has received attention, claims about individual participants are much more challenging. We frame the problem in terms of multiple hypothesis testing: each individual has a null hypothesis (stating that the potential outcomes are equal, for example) and we aim to identify those for whom the null is false (the treatment potential outcome stochastically dominates the control one, for example). We develop a novel algorithm that identifies such a subset, with nonasymptotic control of the false discovery rate (FDR). Our algorithm allows for interaction -- a human data scientist (or a computer program) may adaptively guide the algorithm in a data-dependent manner to gain power. We show how to extend the methods to observational settings and achieve a type of doubly-robust FDR control. We also propose several extensions: (a) relaxing the null to nonpositive effects, (b) moving from unpaired to paired samples, and (c) subgroup identification. We demonstrate via numerical experiments and theoretical analysis that the proposed method has valid FDR control in finite samples and reasonably high identification power.