Optimizing the tie-breaker regression discontinuity design

TL;DR

This paper analyzes tie-breaker designs combining RCT and RDD, quantifies their efficiency, and explores optimal tradeoffs for treatment allocation in various models.

Contribution

It provides a quantitative framework for evaluating and optimizing tie-breaker designs, including efficiency analysis and alternative configurations.

Findings

Efficiency increases with the proportion in RCT, maximized at full RCT.

Tie-breakers reduce boundary bias and variance compared to RDD.

Optimal tradeoffs depend on short-term value versus exploration goals.

Abstract

Motivated by customer loyalty plans and scholarship programs, we study tie-breaker designs which are hybrids of randomized controlled trials (RCTs) and regression discontinuity designs (RDDs). We quantify the statistical efficiency of a tie-breaker design in which a proportion $\Delta$ of observed subjects are in the RCT. In a two line regression, statistical efficiency increases monotonically with $\Delta$, so efficiency is maximized by an RCT. We point to additional advantages of tie-breakers versus RDD: for a nonparametric regression the boundary bias is much less severe and for quadratic regression, the variance is greatly reduced. For a two line model we can quantify the short term value of the treatment allocation and this comparison favors smaller $\Delta$ with the RDD being best. We solve for the optimal tradeoff between these exploration and exploitation goals. The usual tie-breaker design applies an RCT on the middle $\Delta$ subjects as ranked by the assignment variable. We quantify the efficiency of other designs such as experimenting only in the second decile from the top. We also show that in some general parametric models a Monte Carlo evaluation can be replaced by matrix algebra.