A Bootstrap Based Between-Study Heterogeneity Test in Meta-Analysis

TL;DR

This paper introduces bootstrap-based heterogeneity tests for meta-analysis, improving control over Type I errors and power, especially for testing if heterogeneity exceeds a specific level, with practical implementation in an R package.

Contribution

It proposes novel bootstrap methods (B-REML-LRT and B-Q) for heterogeneity testing in meta-analysis, addressing limitations of existing tests in error control and power.

Findings

B-Q outperforms existing tests in simulations.

Proposed methods effectively control Type I error.

An R package is provided for implementation.

Abstract

Meta-analysis combines pertinent information from existing studies to provide an overall estimate of population parameters/effect sizes, as well as to quantify and explain the differences between studies. However, testing the between-study heterogeneity is one of the most troublesome topics in meta-analysis research. Additionally, no methods have been proposed to test whether the size of the heterogeneity is larger than a specific level. The existing methods, such as the Q test and likelihood ratio (LR) tests, are criticized for their failure to control the Type I error rate and/or failure to attain enough statistical power. Although better reference distribution approximations have been proposed in the literature, the expression is complicated and the application is limited. In this article, we propose bootstrap based heterogeneity tests combining the restricted maximum likelihood (REML) ratio test or Q test with bootstrap procedures, denoted as B-REML-LRT and B-Q respectively. Simulation studies were conducted to examine and compare the performance of the proposed methods with the regular LR tests, the regular Q test, and the improved Q test in both the random-effects meta-analysis and mixed-effects meta-analysis. Based on the results of Type I error rates and statistical power, B-Q is recommended. An R package \mathtt{boot.heterogeneity} is provided to facilitate the implementation of the proposed method.