Effect or Treatment Heterogeneity? Policy Evaluation with Aggregated and Disaggregated Treatments

TL;DR

This paper introduces a decomposition method to identify hidden heterogeneity in treatment effects, accounting for multiple treatment versions and confounders, using advanced machine learning techniques.

Contribution

It provides a novel approach to distinguish effect heterogeneity from treatment heterogeneity, applicable in high-dimensional settings with multiple treatments and confounders.

Findings

Heterogeneous effects of smoking on birthweight are partly due to smoking intensities.

Gender gaps in Job Corps effectiveness are largely driven by differential selection.

The method effectively uncovers masked heterogeneity and evaluates treatment assignment quality.

Abstract

Binary treatments are often ex-post aggregates of multiple treatments or can be disaggregated into multiple treatment versions. Thus, effects can be heterogeneous due to either effect or treatment heterogeneity. We propose a decomposition method that uncovers masked heterogeneity, avoids spurious discoveries, and evaluates treatment assignment quality. The estimation and inference procedure based on double/debiased machine learning allows for high-dimensional confounding, many treatments and extreme propensity scores. Our applications suggest that heterogeneous effects of smoking on birthweight are partially due to different smoking intensities and that gender gaps in Job Corps effectiveness are largely explained by differential selection into vocational training.