A General Stability Approach to False Discovery Rate Control

TL;DR

This paper introduces FDR Stabilizer, a general method to improve the stability and reproducibility of false discovery rate control procedures in feature selection, ensuring finite-sample FDR control and asymptotic stability.

Contribution

It proposes a novel stability approach that aggregates multiple runs of FDR procedures, constructs stabilized e-values, and provides theoretical guarantees for FDR control and stability.

Findings

Outperforms existing FDR control methods in experiments

Provides finite-sample FDR bounds and asymptotic stability

Demonstrates improved reproducibility in real data applications

Abstract

Stability and reproducibility are essential considerations in various applications of statistical methods. False Discovery Rate (FDR) control methods are able to control false signals in scientific discoveries. However, many FDR control methods, such as Model-X knockoff and data-splitting approaches, yield unstable results due to the inherent randomness of the algorithms. To enhance the stability and reproducibility of statistical outcomes, we propose a general stability approach for FDR control in feature selection and multiple testing problems, named FDR Stabilizer. Taking feature selection as an example, our method first aggregates feature importance statistics obtained by multiple runs of the base FDR control procedure into a consensus ranking. Then, we construct a stabilized relaxed e-value for each feature and apply the e-BH procedure to these stabilized e-values to obtain the final selection set. We theoretically derive the finite-sample bounds for the FDR and the power of our method, and show that our method asymptotically controls the FDR without power loss. Moreover, we establish the stability of the proposed method, showing that the stabilized selection set converges to a deterministic limit as the number of repetitions increases. Extensive numerical experiments and applications to real datasets demonstrate that the proposed method generally outperforms existing alternatives.