What makes a study design quasi-experimental? The case of difference-in-differences

TL;DR

This paper clarifies the criteria that define quasi-experimental study designs, emphasizing the importance of uncontroversial assumptions over the ability to control for unobserved confounding, using difference-in-differences as an example.

Contribution

It distinguishes two definitions of quasi-experiments and argues that only the one based on uncontroversial assumptions enhances credibility.

Findings

Clarifies ambiguity in quasi-experimental definitions

Differentiates between assumptions-based and confounding-control-based definitions

Uses difference-in-differences to illustrate the distinctions

Abstract

Study designs classified as quasi- or natural experiments are typically accorded more face validity than observational study designs more broadly. However, there is ambiguity in the literature about what qualifies as a quasi-experiment. Here, we attempt to resolve this ambiguity by distinguishing two different ways of defining this term. One definition is based on identifying assumptions being uncontroversial, and the other is based on the ability to account for unobserved sources of confounding (under assumptions). We argue that only the former deserves an additional measure of credibility for reasons of design. We use the difference-in-differences approach to illustrate our discussion.