Assessing the Sensitivity of Synthetic Control Treatment Effect Estimates to Misspecification Error

TL;DR

This paper introduces a sensitivity analysis method for Synthetic Control estimates to evaluate how robust treatment effect conclusions are to potential model misspecification, enhancing the credibility of causal inferences.

Contribution

It presents a data-driven, geometric approach to assess the validity of Synthetic Control estimates by analyzing misspecification error, which is novel in the context of treatment effect evaluation.

Findings

Broadly confirms original study conclusions

Provides a practical tool for robustness assessment

Enhances credibility of Synthetic Control estimates

Abstract

We propose a sensitivity analysis for Synthetic Control (SC) treatment effect estimates to interrogate the assumption that the SC method is well-specified, namely that choosing weights to minimize pre-treatment prediction error yields accurate predictions of counterfactual post-treatment outcomes. Our data-driven procedure recovers the set of treatment effects consistent with the assumption that the misspecification error incurred by the SC method is at most the observable misspecification error incurred when using the SC estimator to predict the outcomes of some control unit. We show that under one definition of misspecification error, our procedure provides a simple, geometric motivation for comparing the estimated treatment effect to the distribution of placebo residuals to assess estimate credibility. When we apply our procedure to several canonical studies that report SC estimates, we broadly confirm the conclusions drawn by the source papers.