Small Study Regression Discontinuity Designs: Density Inclusive Study Size Metric and Performance

TL;DR

This paper introduces the DISS metric for better assessing small RD studies by including running variable density, demonstrating its usefulness through simulations and real school accountability data analysis.

Contribution

It proposes the density inclusive study size (DISS) metric for small RD designs, improving study size characterization by accounting for running variable density.

Findings

DISS improves understanding of RD study power in small samples.

Simulation results show DISS's effectiveness in method comparison.

Application to Indiana school data illustrates practical utility.

Abstract

Regression discontinuity (RD) designs are popular quasi-experimental studies in which treatment assignment depends on whether the value of a running variable exceeds a cutoff. RD designs are increasingly popular in educational applications due to the prevalence of cutoff-based interventions. In such applications sample sizes can be relatively small or there may be sparsity around the cutoff. We propose a metric, density inclusive study size (DISS), that characterizes the size of an RD study better than overall sample size by incorporating the density of the running variable. We show the usefulness of this metric in a Monte Carlo simulation study that compares the operating characteristics of popular nonparametric RD estimation methods in small studies. We also apply the DISS metric and RD estimation methods to school accountability data from the state of Indiana.