Separable Effects in Four-Arm and Two-Arm Designs

TL;DR

This paper develops a framework for analyzing separable effects in four-arm and two-arm experimental designs, providing new identification, estimation methods, and falsification tests, with applications in educational policy.

Contribution

It introduces a general framework for separable effects analysis in four-arm and two-arm designs, including estimation strategies and falsification tests, extending prior two-arm analysis methods.

Findings

Proposed estimators perform well in simulations.

Applied framework to educational data on extended time accommodations.

Falsification tests help validate key assumptions.

Abstract

Robins and Richardson (2010) reformulated mediation analysis by decomposing treatments into multiple components and examining separable effects of each component. While this approach is increasingly popular, existing work has analyzed ``two-arm'' data, where components are strictly bundled and manipulated simultaneously. However, in practice, four-arm data where components are assigned independently are often available. For example, testing accommodations might strictly bundle extra time with a separate session or allow them to be assigned separately. To address this distinction, we propose a general framework for analyzing separable effects in four-arm and two-arm designs. This framework provides distinct identification and estimation strategies for each design. For estimation, we utilize efficient influence function estimators coupled with machine learning and cross-fitting techniques. Additionally, we introduce two falsification tests for key identification assumptions required in the two-arm design by leveraging four-arm data. We investigate the performance of the proposed estimators via a simulation study and demonstrate their application by studying the effect of extended time accommodations using data from the National Assessment of Educational Progress. Ultimately, this separable effects analysis enables practitioners to clearly communicate underlying mechanisms and derive informative policy recommendations.