Design-Based Uncertainty for Quasi-Experiments

TL;DR

This paper introduces a new design-based framework for analyzing quasi-experiments, accounting for stochastic treatment assignment and unobserved selection, thereby enhancing causal inference in social science research.

Contribution

It develops a framework that models stochastic treatment assignment with variable probabilities, providing conditions for interpretability and methods for sensitivity analysis.

Findings

Conditions for estimand interpretability established

Biases from violations of assumptions characterized

Framework enables sensitivity analysis for selection concerns

Abstract

Design-based frameworks of uncertainty are frequently used in settings where the treatment is (conditionally) randomly assigned. This paper develops a design-based framework suitable for analyzing quasi-experimental settings in the social sciences, in which the treatment assignment can be viewed as the realization of some stochastic process but there is concern about unobserved selection into treatment. In our framework, treatments are stochastic, but units may differ in their probabilities of receiving treatment, thereby allowing for rich forms of selection. We provide conditions under which the estimands of popular quasi-experimental estimators correspond to interpretable finite-population causal parameters. We characterize the biases and distortions to inference that arise when these conditions are violated. These results can be used to conduct sensitivity analyses when there are concerns about selection into treatment. Taken together, our results establish a rigorous foundation for quasi-experimental analyses that more closely aligns with the way empirical researchers discuss the variation in the data.