Sub-uniformity of harmonic mean p-values

TL;DR

This paper demonstrates that the harmonic mean of dependent p-values is sub-uniform under various dependence structures, highlighting the need for adjustments when merging p-values in multiple hypothesis testing.

Contribution

It establishes the sub-uniformity of the harmonic mean p-value under multiple dependence assumptions, revealing its anti-conservative nature without proper adjustments.

Findings

Harmonic mean p-value is sub-uniform under dependence.

Harmonic mean p-value merging requires a growing multiplier adjustment.

No universal constant multiplier exists for all p-value sets.

Abstract

We obtain several inequalities on the generalized means of dependent p-values. In particular, the weighted harmonic mean of p-values is strictly sub-uniform under several dependence assumptions of p-values, including independence, negative upper orthant dependence, the class of extremal mixture copulas, and some Clayton copulas. Sub-uniformity of the harmonic mean of p-values has an important implication in multiple hypothesis testing: It is statistically invalid (anti-conservative) to merge p-values using the harmonic mean unless a proper threshold or multiplier adjustment is used, and this applies across all significance levels. The required multiplier adjustment on the harmonic mean p-value grows sub-linearly to infinity as the number of p-values increases, and hence there does not exist a constant multiplier that works for any number of p-values, even under independence.