The L\'evy combination test

TL;DR

The paper introduces the Le9vy combination test (LCT), a new method for combining p-values using heavy-tailed Stable distributions, offering robustness, scalability, and theoretical connections to Bayesian tests.

Contribution

It proposes the Le9vy combination test (LCT), exploiting Le9vy stable distributions for improved robustness and power in combined hypothesis testing.

Findings

LCT is more robust under dependence than CCT and HMP.

LCT controls familywise error rate with higher power than Bonferroni.

LCT's power is intermediate between HMP and Bonferroni.

Abstract

A novel class of methods for combining $p$-values to perform aggregate hypothesis tests has emerged that exploit the properties of heavy-tailed Stable distributions. These methods offer important practical advantages including robustness to dependence and better-than-Bonferroni scaleability, and they reveal theoretical connections between Bayesian and classical hypothesis tests. The harmonic mean $p$-value (HMP) procedure is based on the convergence of summed inverse $p$-values to the Landau distribution, while the Cauchy combination test (CCT) is based on the self-similarity of summed Cauchy-transformed $p$-values. The CCT has the advantage that it is analytic and exact. The HMP has the advantage that it emulates a model-averaged Bayes factor, is insensitive to $p$-values near 1, and offers multilevel testing via a closed testing procedure. Here I investigate whether other Stable combination tests can combine these benefits, and identify a new method, the L\'evy combination test (LCT). The LCT exploits the self-similarity of sums of L\'evy random variables transformed from $p$-values. Under arbitrary dependence, the LCT possesses better robustness than the CCT and HMP, with two-fold worst-case inflation at small significance thresholds. It controls the strong-sense familywise error rate through a multilevel test uniformly more powerful than Bonferroni. Simulations show that the LCT behaves like Simes' test in some respects, with power intermediate between the HMP and Bonferroni. The LCT represents an interesting and attractive addition to combined testing methods based on heavy-tailed distributions.