Admissibility in Partial Conjunction Testing

TL;DR

This paper investigates the conditions under which combined p-values for partial conjunction hypotheses are admissible, proposing a generalized testing method that enhances replicability assessment in meta-analyses.

Contribution

It characterizes admissible combined p-values for partial conjunction testing and introduces a generalized test based on various meta-analysis p-values.

Findings

Non-monotone tests dominate monotone tests in admissibility.

The proposed generalized test improves assessment of replicability.

Application to anticoagulant studies demonstrates practical utility.

Abstract

Meta-analysis combines results from multiple studies aiming to increase power in finding their common effect. It would typically reject the null hypothesis of no effect if any one of the studies shows strong significance. The partial conjunction null hypothesis is rejected only when at least $r$ of $n$ component hypotheses are non-null with $r = 1$ corresponding to a usual meta-analysis. Compared with meta-analysis, it can encourage replicable findings across studies. A by-product of it when applied to different $r$ values is a confidence interval of $r$ quantifying the proportion of non-null studies. Benjamini and Heller (2008) provided a valid test for the partial conjunction null by ignoring the $r - 1$ smallest p-values and applying a valid meta-analysis p-value to the remaining $n - r + 1$ p-values. We provide sufficient and necessary conditions of admissible combined p-value for the partial conjunction hypothesis among monotone tests. Non-monotone tests always dominate monotone tests but are usually too unreasonable to be used in practice. Based on these findings, we propose a generalized form of Benjamini and Heller's test which allows usage of various types of meta-analysis p-values, and apply our method to an example in assessing replicable benefit of new anticoagulants across subgroups of patients for stroke prevention.