The False Discovery Rate for Statistical Pattern Recognition

TL;DR

This paper extends the analysis of false discovery rate (FDR) and false nondiscovery rate (FNDR) to classification, providing finite sample bounds and consistency results for classifiers learned from labeled data.

Contribution

It introduces a novel distribution-free analysis of empirical FDR and FNDR as ratios of binomials, with new bounds and consistency guarantees.

Findings

Derived uniform deviation bounds for FDR and FNDR

Established finite sample bounds for classifiers using FDR and FNDR

Proved strong universal consistency of the proposed methods

Abstract

The false discovery rate (FDR) and false nondiscovery rate (FNDR) have received considerable attention in the literature on multiple testing. These performance measures are also appropriate for classification, and in this work we develop generalization error analyses for FDR and FNDR when learning a classifier from labeled training data. Unlike more conventional classification performance measures, the empirical FDR and FNDR are not binomial random variables but rather a ratio of binomials, which introduces challenges not addressed in conventional analyses. We develop distribution-free uniform deviation bounds and apply these to obtain finite sample bounds and strong universal consistency.