Flexible machine learning estimation of conditional average treatment effects: a blessing and a curse

TL;DR

This paper explores the use of machine learning, specifically causal random forests, to estimate and compare the distributions of individual and conditional average treatment effects, highlighting the importance of variance estimation and underlying assumptions.

Contribution

It extends causal random forests to estimate the difference in variance between treated and control groups, addressing heterogeneity in treatment effects and the limitations of CATE estimation.

Findings

The distributions of ITE and CATE can differ significantly.

Extended CRF can estimate ITE variance when assumptions hold.

Variance differences indicate unmeasured heterogeneity.

Abstract

Causal inference from observational data requires untestable identification assumptions. If these assumptions apply, machine learning (ML) methods can be used to study complex forms of causal effect heterogeneity. Recently, several ML methods were developed to estimate the conditional average treatment effect (CATE). If the features at hand cannot explain all heterogeneity, the individual treatment effects (ITEs) can seriously deviate from the CATE. In this work, we demonstrate how the distributions of the ITE and the CATE can differ when a causal random forest (CRF) is applied. We extend the CRF to estimate the difference in conditional variance between treated and controls. If the ITE distribution equals the CATE distribution, this estimated difference in variance should be small. If they differ, an additional causal assumption is necessary to quantify the heterogeneity not captured by the CATE distribution. The conditional variance of the ITE can be identified when the individual effect is independent of the outcome under no treatment given the measured features. Then, in the cases where the ITE and CATE distributions differ, the extended CRF can appropriately estimate the variance of the ITE distribution while the CRF fails to do so.