Moderating the Mediation Bootstrap for Causal Inference

TL;DR

This paper examines bootstrap methods for causal mediation analysis, revealing their limitations and proposing insights into their coverage properties through simulations and finite-sample distribution analysis.

Contribution

It provides a detailed investigation of bootstrap inference issues in mediation analysis, highlighting the dependence on estimated coefficients and the effects of bias correction.

Findings

Conservative bootstrap behavior is due to dependence on estimated coefficients.

Double bootstrap corrections can be counterproductive in small mediation effects.

Bias-corrected bootstrap can inflate coverage when no mediation is present.

Abstract

Mediation analysis is a form of causal inference that investigates indirect effects and causal mechanisms. Confidence intervals for indirect effects play a central role in conducting inference. The problem is non-standard leading to coverage rates that deviate considerably from their nominal level. The default inference method in the mediation model is the paired bootstrap, which resamples directly from the observed data. However, a residual bootstrap that explicitly exploits the assumed causal structure (X->M->Y) could also be applied. There is also a debate whether the bias-corrected (BC) bootstrap method is superior to the percentile method, with the former showing liberal behavior (actual coverage too low) in certain circumstances. Moreover, bootstrap methods tend to be very conservative (coverage higher than required) when mediation effects are small. Finally, iterated bootstrap methods like the double bootstrap have not been considered due to their high computational demands. We investigate the issues mentioned in the simple mediation model by a large-scale simulation. Results are explained using graphical methods and the newly derived finite-sample distribution. The main findings are: (i) conservative behavior of the bootstrap is caused by extreme dependence of the bootstrap distribution's shape on the estimated coefficients (ii) this dependence leads to counterproductive correction of the the double bootstrap. The added randomness of the BC method inflates the coverage in the absence of mediation, but still leads to (invalid) liberal inference when the mediation effect is small.