When Do Outcome Driven Treatments Break Parallel Trends?

TL;DR

This paper investigates when the parallel trends assumption in Difference-in-Differences studies is violated due to outcome-driven treatment decisions, highlighting the limitations of DiD in such contexts through simulation analyses.

Contribution

It provides a systematic simulation-based assessment of the conditions under which outcome-driven treatments break the parallel trends assumption in DiD studies.

Findings

Parallel trends often do not hold when treatments depend on past outcomes.

Regression to the mean and selection effects explain violations of parallel trends.

DiD may be more appropriate for studying unintended consequences rather than outcome-driven treatments.

Abstract

Under what circumstances is it a threat to the parallel trends assumption required for Difference in Differences (DiD) studies if treatment decisions are based on past values of the outcome? We explore via simulation studies whether parallel trends holds across a grid of data generating processes generally conducive to parallel trends (random walk, Hidden Markov Model, and constant direct additive confounding), study designs (never treated, not yet treated, or later treated control groups), and outcome responsiveness of treatment (yes or no). We interpret the upshot of our simulation results to be that parallel trends is typically not a credible assumption when treatments are influenced by past outcomes. This is due to a combination of regression to the mean and selection on future treatment values, depending on the control group. Since timing of treatment initiation is frequently influenced by past outcomes when the treatment is targeted at the outcome, perhaps DiD is generally better suited for studying unintended consequences of interventions?