Optimal P-value Weighting with Independent Information

TL;DR

This paper introduces a covariate-based weighting method for p-values in large-scale testing, improving detection power especially when true effects are rare and small, by leveraging independent covariate information.

Contribution

It proposes a novel covariate-based weighting approach that optimally incorporates independent information to enhance power in multiple testing, outperforming existing methods in challenging scenarios.

Findings

Outperforms existing methods in rare/low effect size scenarios

Achieves comparable performance in other scenarios

Demonstrates effectiveness through simulations and real data applications

Abstract

The large-scale multiple testing inherent to high throughput biological data necessitates very high statistical stringency and thus true effects in data are difficult to detect unless they have high effect sizes. One solution to this problem is to use an independent information to prioritize the most promising features of the data and thus increase the power to detect them. Weighted p-values provide a general framework for doing this in a statistically rigorous fashion. However, calculating weights that incorporate the independent information and optimize statistical power remains a challenging problem despite recent advances in this area. Existing methods tend to perform poorly in the common situation that true positive features are rare and of low effect size. We introduce covariate based weighting methods for calculating optimal weights conditioned on the effect sizes of the tests. This approach uses the probabilistic relationship between covariate and test effect size to calculate more informative weights that are not diluted by null effects as is common with group-based methods. This relationship can be calculated theoretically for normally distributed covariates or estimated empirically in other cases. We showed via simulations and applications to data that this method outperforms existing methods by a large margin in the rare/low effect size scenario and has at least comparable performance in all scenarios.