Causal Panel Analysis under Parallel Trends: Lessons from a Large Reanalysis Study

TL;DR

This study reanalyzed 49 political science papers using advanced estimators to assess the reliability of causal inferences from panel data, highlighting issues with heterogeneity, assumptions, and study power.

Contribution

It provides a comprehensive reanalysis of existing studies, evaluating the robustness of TWFE models under heterogeneity and parallel trends violations, and offers practical guidance.

Findings

HTE-robust estimators produce similar but variable results

Many studies lack evidence for parallel trends violations

Most studies are underpowered when testing assumptions

Abstract

Two-way fixed effects (TWFE) models are widely used in political science to establish causality, but recent methodological discussions highlight their limitations under heterogeneous treatment effects (HTE) and violations of the parallel trends (PT) assumption. This growing literature has introduced numerous new estimators and procedures, causing confusion among researchers about the reliability of existing results and best practices. To address these concerns, we replicated and reanalyzed 49 studies from leading journals that employ TWFE models for causal inference using observational panel data with binary treatments. Using six HTE-robust estimators, diagnostic tests, and sensitivity analyses, we find: (i) HTE-robust estimators yield qualitatively similar but highly variable results; (ii) while a few studies show clear signs of PT violations, many lack evidence to support this assumption; and (iii) many studies are underpowered when accounting for HTE and potential PT violations. We emphasize the importance of strong research designs and rigorous validation of key identifying assumptions.