Generalized Alpha Investing: Definitions, Optimality Results, and Application to Public Databases

TL;DR

This paper introduces a generalized alpha investing procedure for controlling false discovery rates in large public databases, optimizing power and cost-efficiency compared to previous methods.

Contribution

It develops a more general, optimal version of Alpha Investing for mFDR control and applies it to quality preserving databases to reduce costs and improve false discovery management.

Findings

The generalized procedure outperforms Alpha Investing in power.

Optimal expected reward version enhances false discovery control.

Application to public databases reduces costs significantly.

Abstract

The increasing prevalence and utility of large, public databases necessitates the development of appropriate methods for controlling false discovery. Motivated by this challenge, we discuss the generic problem of testing a possibly infinite stream of null hypotheses. In this context, Foster and Stine (2008) suggested a novel method named Alpha Investing for controlling a false discovery measure known as mFDR. We develop a more general procedure for controlling mFDR, of which Alpha Investing is a special case. We show that in common, practical situations, the general procedure can be optimized to produce an expected reward optimal (ERO) version, which is more powerful than Alpha Investing. We then present the concept of quality preserving databases (QPD), originally introduced in Aharoni et al. (2011), which formalizes efficient public database management to simultaneously save costs and control false discovery. We show how one variant of generalized alpha investing can be used to control mFDR in a QPD and lead to significant reduction in costs compared to naive approaches for controlling the Family-Wise Error Rate implemented in Aharoni et al. (2011).