Simulation study of Q statistic with constant weights for testing and estimation of heterogeneity of standardized mean differences in meta-analysis

TL;DR

This study compares the traditional Cochran's Q statistic with a new constant-weighted Q statistic using simulations to improve heterogeneity testing and estimation of between-study variance in meta-analysis.

Contribution

It introduces a new Q statistic with constant weights based on effective sample sizes and evaluates its performance through simulations for heterogeneity testing and variance estimation.

Findings

Q_F approximates distribution better than Q_IV in simulations

New estimators of tau^2 show improved bias properties

Constant weights simplify heterogeneity testing procedures

Abstract

Cochran's $Q$ statistic is routinely used for testing heterogeneity in meta-analysis. Its expected value is also used for estimation of between-study variance $\tau^2$. Cochran's $Q$, or $Q_{IV}$, uses estimated inverse-variance weights which makes approximating its distribution rather complicated. As an alternative, we are investigating a new $Q$ statistic, $Q_F$, whose constant weights use only the studies' effective sample sizes. For standardized mean difference as the measure of effect, we study, by simulation, approximations to distributions of $Q_{IV}$ and $Q_F$, as the basis for tests of heterogeneity and for new point and interval estimators of the between-study variance $\tau^2$. These include new DerSimonian-Kacker (2007)-type moment estimators based on the first moment of $Q_F$, and novel median-unbiased estimators of $\tau^2$.