Weighted Hypothesis Testing

TL;DR

This paper develops optimal weighting strategies for multiple hypothesis testing, demonstrating robustness to weight misspecification and proposing methods based on prior information and data estimation.

Contribution

It derives the optimal weights for weighted p-value testing and compares external and estimated weighting methods for practical application.

Findings

Optimal weights significantly improve testing power.

Power is robust to inaccuracies in weight specification.

Data-driven weighting performs comparably to prior-based methods.

Abstract

The power of multiple testing procedures can be increased by using weighted p-values (Genovese, Roeder and Wasserman 2005). We derive the optimal weights and we show that the power is remarkably robust to misspecification of these weights. We consider two methods for choosing weights in practice. The first, external weighting, is based on prior information. The second, estimated weighting, uses the data to choose weights.