Choosing Between Methods of Combining p-values

TL;DR

This paper analyzes various methods for combining p-values in meta-analysis, providing theoretical guidance on selecting the most powerful approach based on likelihood ratio test perspectives.

Contribution

It recasts p-value combination methods as likelihood ratio tests and offers theoretical insights to guide the choice of the most effective method.

Findings

Likelihood ratio test framework for p-value combiners

Theoretical guidance for selecting powerful p-value combination methods

Analysis of standard p-value combining techniques

Abstract

Combining p-values from independent statistical tests is a popular approach to meta-analysis, particularly when the data underlying the tests are either no longer available or are difficult to combine. A diverse range of p-value combination methods appear in the literature, each with different statistical properties. Yet all too often the final choice used in a meta-analysis can appear arbitrary, as if all effort has been expended building the models that gave rise to the p-values. Birnbaum (1954) showed that any reasonable p-value combiner must be optimal against some alternative hypothesis. Starting from this perspective and recasting each method of combining p-values as a likelihood ratio test, we present theoretical results for some of the standard combiners which provide guidance about how a powerful combiner might be chosen in practice.