Evaluating Counterfactual Policies Using Instruments

TL;DR

This paper introduces a flexible framework for evaluating the effects of counterfactual policies using instrumental variables, avoiding strong assumptions and providing sharp bounds on policy impacts.

Contribution

It develops a computationally tractable method to bound policy effects with IVs without relying on the IV monotonicity assumption, and explores alternative restrictions.

Findings

Bounds are often tight without IV monotonicity.

Policy invariance assumptions do not tighten bounds in certain cases.

Application to bail judge assignments demonstrates practical utility.

Abstract

We study settings in which a researcher has an instrumental variable (IV) and seeks to evaluate the effects of a counterfactual policy that alters treatment assignment, such as a directive encouraging randomly assigned judges to release more defendants. We develop a general and computationally tractable framework for computing sharp bounds on the effects of such policies. Our approach does not require the often tenuous IV monotonicity assumption. Moreover, for an important class of policy exercises, we show that IV monotonicity -- while crucial for a causal interpretation of two-stage least squares -- does not tighten the bounds on the counterfactual policy impact. We analyze the identifying power of alternative restrictions, including the policy invariance assumption used in the marginal treatment effect literature, and develop a relaxation of this assumption. We illustrate our framework using applications to quasi-random assignment of bail judges in New York City and prosecutors in Massachusetts.