Randomization Inference for Before-and-After Studies with Multiple Units: An Application to a Criminal Procedure Reform in Uruguay

TL;DR

This paper introduces a randomization inference method for short-term causal analysis in before-and-after policy studies with multiple units, demonstrated through a criminal justice reform in Uruguay, providing robust, non-parametric statistical conclusions.

Contribution

It develops a novel randomization inference framework tailored for short-term causal effects in multi-unit before-and-after studies, avoiding parametric assumptions.

Findings

Significant increase of 25 police reports in the first week post-reform

Method validated with falsification tests supporting no local time trends

Framework applicable for policy evaluation with multiple units

Abstract

Learning about the immediate causal effects of large-scale policy interventions poses a significant challenge for quasi-experimental methods that rely on long-term trends or parametric modeling assumptions. As an alternative, we develop a randomization inference framework for before-and-after studies with multiple units, designed specifically for short-term causal inference and allowing for general assignment mechanisms. The method provides finite-sample-valid statistical inferences without relying on parametric time series models or extrapolation. We demonstrate its utility by analyzing a major criminal justice reform in Uruguay that switched from an inquisitorial to an adversarial system in November 2017. Our method relies on the key assumption of no local time trends near the policy adoption time, which is supported by several falsification tests in our empirical study. We find a statistically significant short-term causal effect: an increase of approximately 25 daily police reports (an 8% rise) in the first week of the new justice system. Our randomization inference framework provides a robust and flexible methodology for evaluating policy adoptions in before-and-after studies with multiple units.