Synthetic Control Misconceptions: Recommendations for Practice

TL;DR

This paper critically examines common misconceptions about synthetic control methods, revealing that some widely accepted practices are unsupported by empirical evidence, and offers practical guidelines for better implementation.

Contribution

It identifies prevalent misconceptions in synthetic control methodology and provides evidence-based recommendations to improve its application and interpretation.

Findings

Some common beliefs about robustness are unsupported by simulations

Covariates are more important than often assumed

Pre-treatment prediction error is not always a reliable guide

Abstract

To estimate the causal effect of an intervention, researchers need to identify a control group that represents what might have happened to the treatment group in the absence of that intervention. This is challenging without a randomized experiment and further complicated when few units (possibly only one) are treated. Nevertheless, when data are available on units over time, synthetic control (SC) methods provide an opportunity to construct a valid comparison by differentially weighting control units that did not receive the treatment so that their resulting pre-treatment trajectory is similar to that of the treated unit. The hope is that this weighted ``pseudo-counterfactual" can serve as a valid counterfactual in the post-treatment time period. Since its origin twenty years ago, SC has been used over 5,000 times in the literature (Web of Science, December 2025), leading to a proliferation of descriptions of the method and guidance on proper usage that is not always accurate and does not always align with what the original developers appear to have intended. As such, a number of accepted pieces of wisdom have arisen: (1) SC is robust to various implementations; (2) covariates are unnecessary, and (3) pre-treatment prediction error should guide model selection. We describe each in detail and conduct simulations that suggest, both for standard and alternative implementations of SC, that these purported truths are not supported by empirical evidence and thus actually represent misconceptions about best practice. Instead of relying on these misconceptions, we offer practical advice for more cautious implementation and interpretation of results.