Interpreting OLS Estimands When Treatment Effects Are Heterogeneous: Smaller Groups Get Larger Weights

TL;DR

This paper examines how OLS estimates of treatment effects are weighted when effects are heterogeneous, revealing that smaller groups receive larger weights, which can bias interpretations in applied research.

Contribution

It demonstrates that OLS treatment coefficients are weighted averages of subgroup effects, with weights inversely related to group sizes, and provides diagnostic tools to identify potential biases.

Findings

OLS estimand is a convex combination of subgroup effects.

Smaller groups receive larger weights in the estimation.

Diagnostic tools help detect and correct biases in applied analysis.

Abstract

Applied work often studies the effect of a binary variable ("treatment") using linear models with additive effects. I study the interpretation of the OLS estimands in such models when treatment effects are heterogeneous. I show that the treatment coefficient is a convex combination of two parameters, which under certain conditions can be interpreted as the average treatment effects on the treated and untreated. The weights on these parameters are inversely related to the proportion of observations in each group. Reliance on these implicit weights can have serious consequences for applied work, as I illustrate with two well-known applications. I develop simple diagnostic tools that empirical researchers can use to avoid potential biases. Software for implementing these methods is available in R and Stata. In an important special case, my diagnostics only require the knowledge of the proportion of treated units.