Individual Treatment Effect: Prediction Intervals and Sharp Bounds

TL;DR

This paper explores the fundamental limits of inferring individual treatment effects from randomized trial data, providing bounds, prediction intervals, and contrasting them with average treatment effect inference, highlighting challenges in personalized decision-making.

Contribution

It characterizes prediction intervals and sharp bounds for ITE under partial identification, extending to various outcome types and contrasting with ATE confidence intervals.

Findings

Prediction intervals for ITE are generally wider and do not vanish with larger samples.

Sharp bounds on the ITE pmf are derived for binary, continuous, and ordinal outcomes.

Contrasts between ITE prediction intervals and ATE confidence intervals reveal fundamental inferential differences.

Abstract

Individual treatment effect (ITE) is often regarded as the ideal target of inference in causal analyses and has been the focus of several recent studies. In this paper, we describe the intrinsic limits regarding what can be learned concerning ITEs given data from large randomized experiments. We consider when a valid prediction interval for the ITE is informative and when it can be bounded away from zero. The joint distribution over potential outcomes is only partially identified from a randomized trial. Consequently, to be valid, an ITE prediction interval must be valid for all joint distribution consistent with the observed data and hence will in general be wider than that resulting from knowledge of this joint distribution. We characterize prediction intervals in the binary treatment and outcome setting, and extend these insights to models with continuous and ordinal outcomes. We derive sharp bounds on the probability mass function (pmf) of the individual treatment effect (ITE). Finally, we contrast prediction intervals for the ITE and confidence intervals for the average treatment effect (ATE). This also leads to the consideration of Fisher versus Neyman null hypotheses. While confidence intervals for the ATE shrink with increasing sample size due to its status as a population parameter, prediction intervals for the ITE generally do not vanish, leading to scenarios where one may reject the Neyman null yet still find evidence consistent with the Fisher null, highlighting the challenges of individualized decision-making under partial identification.