Bias-Variance Tradeoff of Matching Prior to Difference-in-Differences When Parallel Trends is Violated

TL;DR

This paper analyzes the bias and variance tradeoffs of matching prior to Difference-in-Differences (DiD) when the parallel trends assumption is violated, providing guidelines for practitioners.

Contribution

It extends bias analysis to include variance and MSE, offering new insights and practical guidelines for matching strategies in DiD under weaker assumptions.

Findings

Matching on observed covariates may not always reduce MSE due to sample size tradeoff.

Matching on pre-treatment outcomes consistently improves MSE.

Guidelines are provided for practitioners on when and what variables to match.

Abstract

Quasi-experimental causal inference methods have become central in empirical operations management for guiding managerial decisions. Among these, empiricists utilize the Difference-in-Differences (DiD) estimator, which relies on the parallel trends assumption. To improve its plausibility, researchers often match treated and control units before applying DiD, with the intuition that matched groups are more likely to evolve similarly absent treatment. Existing work that analyzes this practice, however, has focused solely on bias. In this work, we not only generalize earlier bias results under weaker assumptions but also analyze properties of variance and mean squared error (MSE), a practically relevant metric for decision making. Under a linear structural model with unobserved time-varying confounders, we show that variance results contrast with established bias insights: matching on observed covariates prior to DiD is not always recommended over the classic (unmatched) DiD due to a sample size tradeoff; furthermore, matching additionally on pre-treatment outcomes is always beneficial as such tradeoff no longer exists once matching is performed. We therefore advocate MSE as an additional metric if applied researchers weigh bias and variance equally and further give practitioner-friendly guidelines with theoretical guarantees on when and on what variables they should match. As an illustration, we apply these guidelines to re-examine a recent empirical study that matches prior to DiD to study how the introduction of monetary incentives by a knowledge-sharing platform affects general engagement on the platform. Our results show that the authors' decision was both warranted and critical to produce a credible causal estimate.