Valid Inference when Testing Violations of Parallel Trends for Difference-in-Differences

TL;DR

This paper introduces simple, consistent preliminary tests and confidence intervals for difference-in-differences analysis that remain valid under violations of the parallel trends assumption, improving causal inference.

Contribution

It proposes new tests and confidence intervals that address low power and bias issues in existing methods, under mild separation and conditional extrapolation assumptions.

Findings

Proposed tests are consistent under mild separation conditions.

Confidence intervals have valid coverage conditional on passing the test.

Methods perform well on synthetic and real data examples.

Abstract

The difference-in-differences (DID) research design is a key identification strategy which allows researchers to estimate causal effects under the parallel trends assumption. While the parallel trends assumption is counterfactual and cannot be tested directly, researchers often examine pre-treatment periods to check whether the time trends are parallel before treatment is administered. A recent literature has shown that existing preliminary tests have adverse effects on conventional statistical methods for estimation and inference, including low power, bias, and undercoverage. In this paper, we describe simple preliminary tests and corresponding confidence intervals for the causal effect which overcome these issues. Under mild separation conditions, the preliminary test is shown to be consistent and the confidence intervals for the causal effect have valid coverage conditional on passing the test. Our results hold under what we refer to as the conditional extrapolation assumption, which posits a relationship between the unidentified post-treatment violation of parallel trends and the identified pre-treatment violations. We view the conditional extrapolation assumption as one formalization of the assumption which is implicitly held when conducting a preliminary test for parallel trends. To illustrate the performance of the proposed methods, we use synthetic data as well as data on recentralization of public services in Vietnam and right-to-carry laws in Virginia.