On ratio measures of population heterogeneity for meta-analyses

TL;DR

This paper critiques common heterogeneity measures in meta-analyses, proposes variants of the coefficient of variation for better interpretability, and introduces interval estimators with strong coverage properties, validated through simulations.

Contribution

It introduces interpretable CV variants for heterogeneity measurement and provides reliable interval estimators with strong coverage, improving meta-analysis assessments.

Findings

CV variants in (0, 1] are easier to interpret.

Interval estimators show excellent coverage in simulations.

Proposed methods outperform existing heterogeneity measures.

Abstract

Popular measures of meta-analysis heterogeneity, such as $I^2$, cannot be considered measures of population heterogeneity since they are dependant on samples sizes within studies. The coefficient of variation (CV) recently introduced and defined to be the heterogeneity variance divided by the absolute value of the overall mean effect does not suffer such shortcomings. However, very large CV values can occur when the effect is small making interpretation difficult. The purpose of this paper is two-fold. Firstly, we consider variants of the CV that exist in the interval (0, 1] making interpretation simpler. Secondly, we provide interval estimators for the CV and its variants with excellent coverage properties. We perform simulation studies based on simulated and real data sets and draw comparisons between the methods.