A Bayesian Classification Trees Approach to Treatment Effect Variation with Noncompliance

TL;DR

This paper introduces a Bayesian Causal Forest model to estimate heterogeneous treatment effects in randomized trials with noncompliance, effectively addressing weak instrument issues and revealing nuanced treatment impacts.

Contribution

The paper presents a novel Bayesian approach for estimating treatment effect heterogeneity in noncompliance scenarios, improving flexibility and robustness over existing methods.

Findings

Effective detection of heterogeneity in treatment effects.

Null average effect on chronic condition presence.

Significant heterogeneity in perceptions of management prioritization.

Abstract

Estimating varying treatment effects in randomized trials with noncompliance is inherently challenging since variation comes from two separate sources: variation in the impact itself and variation in the compliance rate. In this setting, existing flexible machine learning methods are highly sensitive to the weak instruments problem, in which the compliance rate is (locally) close to zero. Our main methodological contribution is to present a Bayesian Causal Forest model for binary response variables in scenarios with noncompliance. By repeatedly imputing individuals' compliance types, we can flexibly estimate heterogeneous treatment effects among compliers. Simulation studies demonstrate the usefulness of our approach when compliance and treatment effects are heterogeneous. We apply the method to detect and analyze heterogeneity in the treatment effects in the Illinois Workplace Wellness Study, which not only features heterogeneous and one-sided compliance but also several binary outcomes of interest. We demonstrate the methodology on three outcomes one year after intervention. We confirm a null effect on the presence of a chronic condition, discover meaningful heterogeneity impact of the intervention on metabolic parameters though the average effect is null in classical partial effect estimates, and find substantial heterogeneity in individuals' perception of management prioritization of health and safety.