Interactive Martingale Tests for the Global Null

TL;DR

This paper introduces martingale-based global null tests suitable for online and batch settings, demonstrating higher power and adaptability through theory, simulations, and interactive covariate use.

Contribution

It develops simple martingale analogs of classical tests for global null, including an interactive, data-adaptive test leveraging covariates and user guidance.

Findings

Martingale tests outperform classical tests in power.

Interactive test adapts to structured alternatives.

Higher detection power achieved with covariate integration.

Abstract

Global null testing is a classical problem going back about a century to Fisher's and Stouffer's combination tests. In this work, we present simple martingale analogs of these classical tests, which are applicable in two distinct settings: (a) the online setting in which there is a possibly infinite sequence of $p$-values, and (b) the batch setting, where one uses prior knowledge to preorder the hypotheses. Through theory and simulations, we demonstrate that our martingale variants have higher power than their classical counterparts even when the preordering is only weakly informative. Finally, using a recent idea of "masking" $p$-values, we develop a novel interactive test for the global null that can take advantage of covariates and repeated user guidance to create a data-adaptive ordering that achieves higher detection power against structured alternatives.