Combining p-values via averaging

TL;DR

This paper introduces new methods for combining multiple p-values in hypothesis testing without dependence assumptions, extending classical results to generalized means and discussing their efficiency and application.

Contribution

It extends classical p-value combination methods to generalized means using recent mathematical finance techniques, including weighted averages and efficiency considerations.

Findings

Harmonic mean combination scaled by ln K is effective for large K.

Generalized means provide flexible p-value combination strategies.

The methods improve robustness in multiple testing scenarios.

Abstract

This paper proposes general methods for the problem of multiple testing of a single hypothesis, with a standard goal of combining a number of p-values without making any assumptions about their dependence structure. An old result by R\"uschendorf and, independently, Meng implies that the p-values can be combined by scaling up their arithmetic mean by a factor of 2 (and no smaller factor is sufficient in general). A similar result about the geometric mean (Mattner) replaces 2 by $e$. Based on more recent developments in mathematical finance, specifically, robust risk aggregation techniques, we extend these results to generalized means; in particular, we show that $K$ p-values can be combined by scaling up their harmonic mean by a factor of $\ln K$ (asymptotically as $K\to\infty$). This leads to a generalized version of the Bonferroni-Holm procedure. We also explore methods using weighted averages of p-values. Finally, we discuss the efficiency of various methods of combining p-values and how to choose a suitable method in light of data and prior information.