Counting Defiers: A Design-Based Model of an Experiment Can Reveal Evidence Beyond the Average Effect

TL;DR

This paper introduces a design-based likelihood approach to identify and analyze defiers in randomized experiments, revealing evidence beyond average effects by examining joint potential outcomes.

Contribution

It develops a novel likelihood framework that accounts for defiers, providing new insights into treatment effect heterogeneity beyond traditional methods.

Findings

Samples with defiers can generate more data configurations.

Likelihood varies within Frechet bounds, unlike traditional likelihood.

Evidence for defiers exists but is weak, demonstrated with health data.

Abstract

Using only a binary intervention and outcome and the design of the randomization within an experiment, we construct a design-based likelihood of the joint distribution of potential outcomes in the sample -- the numbers of always takers, compliers, defiers, and never takers. We develop a visualization to show that samples with defiers can sometimes generate the data in more ways than samples without, yielding a higher likelihood. This likelihood can vary within the Frechet bounds, even though the traditional likelihood does not. Evidence is weak, but it exists, as we illustrate with health applications and our dbmle package.