Meta-analysis of ratios of sample variances

TL;DR

This paper introduces a meta-analytic method for estimating variance ratios across studies using F-test statistics, aiding in assessing the validity of equal variance assumptions in meta-analyses of standardized mean differences.

Contribution

It proposes a novel meta-analytic approach to estimate variance ratios, incorporating variance stabilization and maximum likelihood estimators to improve variance assessment.

Findings

The method allows inclusion of studies with potential variance inequality.

Provides confidence intervals for variance ratios to guide analysis decisions.

Enables visual inspection of variance assumptions through QQ-plots.

Abstract

When conducting a meta-analysis of standardized mean differences (SMDs), it is common to assume equal variances in the two arms of each study. This leads to Cohen's $d$ estimates for which interpretation is simple. However, this simplicity should not be used as a justification for the assumption of equal variances in situations where evidence may suggest that it is incorrect. Until now, researchers have either used an $F$-test for each individual study as a justification for the equality of variances or perhaps even conveniently ignored such tools altogether. In this paper we propose using a meta-analysis of F-test statistics to estimate the ratio of variances prior to the combination of SMD's. This procedure allows some studies to be included that might otherwise be omitted by individual fixed level tests for unequal variances, sometimes occur even when the assumption of equal variances holds. The estimated ratio of variances, as well as associated confidence intervals, can be used as guidance as to whether the assumption of equal variances is violated. The estimators considered include variance stabilization transformations (VST) of the $F$-test statistics as well as MLE estimators. The VST approaches enable the use of QQ-plots to visually inspect for violations of equal variances while the MLE estimator easily allows for the introduction of a random effect. When there is evidence of unequal variances, this work provides a means to formally justify the use of less common methods such as log ratio of means when studies are measured on a different scale.