Identifying and Estimating Principal Causal Effects in Multi-site Trials

TL;DR

This paper explores how to estimate principal causal effects in multi-site randomized trials using the principal stratification framework, highlighting conditions under which cross-site variation can inform about subgroup effects.

Contribution

It introduces a method to estimate principal causal effects across sites assuming site-specific effects are uncorrelated with stratum membership, applied to an education intervention.

Findings

Positive effects for students likely to attend low-quality high schools

Method demonstrates potential for leveraging multi-site variation

Analysis limited by statistical power

Abstract

Randomized trials are often conducted with separate randomizations across multiple sites such as schools, voting districts, or hospitals. These sites can differ in important ways, including the site's implementation, local conditions, and the composition of individuals. An important question in practice is whether---and under what assumptions---researchers can leverage this cross-site variation to learn more about the intervention. We address these questions in the principal stratification framework, which describes causal effects for subgroups defined by post-treatment quantities. We show that researchers can estimate certain principal causal effects via the multi-site design if they are willing to impose the strong assumption that the site-specific effects are uncorrelated with the site-specific distribution of stratum membership. We motivate this approach with a multi-site trial of the Early College High School Initiative, a unique secondary education program with the goal of increasing high school graduation rates and college enrollment. Our analyses corroborate previous studies suggesting that the initiative had positive effects for students who would have otherwise attended a low-quality high school, although power is limited.