Matching $\leq$ Hybrid $\leq$ Difference in Differences

TL;DR

This paper investigates the relative accuracy of Matching, Hybrid, and Difference-in-Differences methods in estimating treatment effects, revealing a consistent inequality relationship supported by theoretical justification.

Contribution

It provides a formal theoretical explanation for the inequality relationship among these methods under certain conditions, enhancing understanding of their comparative performance.

Findings

Matching $\, extless\, ext{Hybrid}\, extless\,DID}$ as a consistent norm

DID tends to be optimistic under non-negative treatment assumptions

Matching offers more conservative estimates

Abstract

Since LaLonde's (1986) seminal paper, there has been ongoing interest in estimating treatment effects using pre- and post-intervention data. Scholars have traditionally used experimental benchmarks to evaluate the accuracy of alternative econometric methods, including Matching, Difference-in-Differences (DID), and their hybrid forms (e.g., Heckman et al., 1998b; Dehejia and Wahba, 2002; Smith and Todd, 2005). We revisit these methodologies in the evaluation of job training and educational programs using four datasets (LaLonde, 1986; Heckman et al., 1998a; Smith and Todd, 2005; Chetty et al., 2014a; Athey et al., 2020), and show that the inequality relationship, Matching $\leq$ Hybrid $\leq$ DID, appears as a consistent norm, rather than a mere coincidence. We provide a formal theoretical justification for this puzzling phenomenon under plausible conditions such as negative selection, by generalizing the classical bracketing (Angrist and Pischke, 2009, Section 5). Consequently, when treatments are expected to be non-negative, DID tends to provide optimistic estimates, while Matching offers more conservative ones.