A new paradox in random-effects meta-analysis

TL;DR

This paper uncovers a paradox in random-effects meta-analysis that arises with few studies or high heterogeneity, questioning the model's validity and urging caution in interpretation.

Contribution

It introduces a novel paradox in random-effects meta-analysis, highlighting potential issues with the model under certain conditions and prompting reconsideration of its appropriateness.

Findings

Paradox occurs with small number of studies or high heterogeneity.

Meta-analysts should interpret results with caution.

Raises questions about the validity of the current random-effects model.

Abstract

Meta-analysis is an important tool for combining results from multiple studies and has been widely used in evidence-based medicine for several decades. This paper reports, for the first time, an interesting and valuable paradox in random-effects meta-analysis that is likely to occur when the number of studies is small and/or the heterogeneity is large. With the incredible paradox, we hence advocate meta-analysts to be extremely cautious when interpreting the final results from the random-effects meta-analysis. And more importantly, with the unexpected dilemma in making decisions, the new paradox has raised an open question whether the current random-effects model is reasonable and tenable for meta-analysis, or it needs to be abandoned or further improved to some extent.