Testing for idiosyncratic Treatment Effect Heterogeneity

TL;DR

This paper introduces a new asymptotically valid statistical test for detecting idiosyncratic treatment effect heterogeneity, overcoming nuisance parameter issues and demonstrating higher power through simulations and a microfinance experiment application.

Contribution

It proposes a novel test based on the empirical characteristic function that effectively detects unobserved heterogeneity in treatment effects, improving upon existing methods.

Findings

The test maintains validity in the presence of nuisance parameters.

Simulation results show higher power compared to current tests.

Application to a microfinance experiment indicates unexplained heterogeneity in outcomes.

Abstract

This paper provides asymptotically valid tests for the null hypothesis of no treatment effect heterogeneity. Importantly, I consider the presence of heterogeneity that is not explained by observed characteristics, or so-called idiosyncratic heterogeneity. When examining this heterogeneity, common statistical tests encounter a nuisance parameter problem in the average treatment effect which renders the asymptotic distribution of the test statistic dependent on that parameter. I propose an asymptotically valid test that circumvents the estimation of that parameter using the empirical characteristic function. A simulation study illustrates not only the test's validity but its higher power in rejecting a false null as compared to current tests. Furthermore, I show the method's usefulness through its application to a microfinance experiment in Bosnia and Herzegovina. In this experiment and for outcomes related to loan take-up and self-employment, the tests suggest that treatment effect heterogeneity does not seem to be completely accounted for by baseline characteristics. For those outcomes, researchers could potentially try to collect more baseline characteristics to inspect the remaining treatment effect heterogeneity, and potentially, improve treatment targeting.