A comparison of combined p-value functions for meta-analysis

TL;DR

This paper systematically compares various p-value combination methods for meta-analysis, highlighting the unique invariance property of Edgington's method and its competitive performance in simulations.

Contribution

It provides a comprehensive theoretical and simulation-based comparison of p-value combination procedures, emphasizing Edgington's method's advantages.

Findings

Many methods are not invariant to p-value orientation

Edgington's method is orientation-invariant and symmetric

Edgington's method performs well in heterogeneity scenarios

Abstract

P-value functions are modern statistical tools that unify effect estimation and hypothesis testing and can provide alternative point and interval estimates compared to standard meta-analysis methods, using any of the many $p$-value combination procedures available (Xie et al., 2011, JASA). We provide a systematic comparison of different combination procedures, both from a theoretical perspective and through simulation. We show that many prominent p-value combination methods (e.g. Fisher's method) are not invariant to the orientation of the underlying one-sided p-values. Only Edgington's method, a lesser-known combination method based on the sum of $p$-values, is orientation-invariant and still provides confidence intervals not restricted to be symmetric around the point estimate. Adjustments for heterogeneity can also be made and results from a simulation study indicate that Edgington's method can compete with more standard meta-analytic methods.