Analysis of Broken Randomized Experiments by Principal Stratification

TL;DR

This paper develops a principal stratification framework to analyze broken randomized experiments, addressing issues like non-compliance and missing data, with applications to training effects on employment.

Contribution

It introduces a new statistical analysis method for broken experiments using principal stratification and proposes an interventionist estimand to relax assumptions.

Findings

Training improves employment outcomes in the Job Corps study.

The estimator's asymptotic properties are established.

The framework effectively handles post-treatment complications.

Abstract

Although randomized controlled trials have long been regarded as the ``gold standard'' for evaluating treatment effects, there is no natural prevention from post-treatment events. For example, non-compliance makes the actual treatment different from the assigned treatment, truncation-by-death renders the outcome undefined or ill-defined, and missingness prevents the outcomes from being measured. In this paper, we develop a statistical analysis framework using principal stratification to investigate the treatment effect in broken randomized experiments. The average treatment effect in compliers and always-survivors is adopted as the target causal estimand. We establish the asymptotic property for the estimator. To relax the identification assumptions, we also propose an interventionist estimand defined in compliers by adjusting for baseline covariates. We apply the framework to study the effect of training on earnings in the Job Corps study and find that the training program improves employment and earnings in the long term.