A Guide to Impact Evaluation under Sample Selection and Missing Data: Teacher's Aides and Adolescent Mental Health

TL;DR

This paper develops a unifying graphical framework for impact evaluation with sample selection and missing data, demonstrating its application in a large-scale teacher's aide trial affecting adolescent mental health.

Contribution

It introduces a novel, comprehensive approach integrating identification and testing strategies for missing data and sample selection in causal evaluation.

Findings

Teaching assistants improve adolescent mental health outcomes.

Handling missing data significantly influences evaluation conclusions.

Framework aids in robust impact assessment under data limitations.

Abstract

This paper is concerned with identification, estimation, and specification testing in causal evaluation problems when data is selective and/or missing. We leverage recent advances in the literature on graphical methods to provide a unifying framework for guiding empirical practice. The approach integrates and connects to prominent identification and testing strategies in the literature on missing data, causal machine learning, panel data analysis, and more. We demonstrate its utility in the context of identification and specification testing in sample selection models and field experiments with attrition. We provide a novel analysis of a large-scale cluster-randomized controlled teacher's aide trial in Danish schools at grade 6. Even with detailed administrative data, the handling of missing data crucially affects broader conclusions about effects on mental health. Results suggest that teaching assistants provide an effective way of improving internalizing behavior for large parts of the student population.