Inference of Sample Complier Average Causal Effects under Experiments with Completely Randomized Design and Computer Assisted Balance-Improving Designs

TL;DR

This paper develops and compares inference methods for estimating the causal effect among compliers in randomized experiments, especially under computer-assisted designs like rerandomization, highlighting Bayesian inference's superior performance.

Contribution

It introduces and evaluates Bayesian, Wald, and regression adjustment estimators for complier average causal effects under various experimental designs, including rerandomization.

Findings

Bayesian method outperforms others in stability and accuracy

Bayesian inference yields smallest median errors and interval lengths

Bayesian advantage is larger with fewer compliers

Abstract

Non-compliance is common in real world experiments. We focus on inference about the sample complier average causal effect, that is, the average treatment effect for experimental units who are compliers. We present three types of inference strategies for the sample complier average causal effect: the Wald estimator, regression adjustment estimators and model-based Bayesian inference. Because modern computer assisted experimental designs have been used to improve covariate balance over complete randomization, we discuss inference under both complete randomization and a specific computer assisted experimental design - Mahalanobis distance based rerandomization, under which asymptotic properties of the Wald estimator and regression adjustment estimators can be derived. We use Monte Carlo simulation to compare the finite sample performance of the methods under both experimental designs. We find that under either design, the Bayesian method performs the best because it is stable, it yields smallest median absolute error and smallest median interval length. The improvement by the Bayesian method is especially large when the fraction of compliers is small. We present an application to a job training experiment with non-compliance.