Confidence envelopes for the false discoveries with heterogeneous data

TL;DR

This paper develops new confidence envelopes for false discoveries in heterogeneous, discrete data, improving upon existing methods designed for homogeneous p-values by incorporating tailored inequalities.

Contribution

It introduces novel confidence envelopes that adapt to heterogeneous data, bridging previous homogeneous methods with new tools like Bretagnolle and Simes inequalities.

Findings

New envelopes outperform homogeneous counterparts in simulations

Tailored inequalities improve false discovery control in heterogeneous data

Proposed methods maintain statistical guarantees with discrete p-values

Abstract

In the context of selective inference, confidence envelopes for the false discoveries allow the user to select any subset of null hypotheses while having a statistical guarantee on the number of false discoveries in the selected set. Many constructions of such envelopes have been proposed recently, using local test families (Genovese and Wasserman, 2006; Goeman and Solari, 2011), paths (Katsevich and Ramdas, 2020) or interpolation (Blanchard et al., 2020a). All those methods have in common that they have been well-studied for the homogeneous case where all p-values under the null have a uniform distribution over [0, 1]. However, in many applications the data are heterogeneous and discrete, hence the p-values have heterogeneous, discrete distributions, and the previous constructions may incur a loss of power, in the sense that they over-estimate the number of false discoveries. In this paper, we bridge the previous constructions under the homogeneous case with new tools. We also apply these tools to propose several confidence envelopes based on tools tailored for heterogeneous data, like the Bretagnolle inequality, or a new variant of the Simes inequality. We compare these new envelopes to their homogeneous counterparts on simulated data.