A simple forward selection procedure based on false discovery rate control

TL;DR

This paper introduces a new FDR-based model selection method that outperforms existing penalized methods across various realistic settings, demonstrating empirical minimax performance especially at an FDR level of 0.05.

Contribution

The paper proposes a novel FDR controlling procedure for model selection and compares its effectiveness to other methods through extensive simulations.

Findings

FDR-based procedures perform well across diverse settings.

The new method shows empirical minimax performance.

FDR level of 0.05 yields optimal results.

Abstract

We propose the use of a new false discovery rate (FDR) controlling procedure as a model selection penalized method, and compare its performance to that of other penalized methods over a wide range of realistic settings: nonorthogonal design matrices, moderate and large pool of explanatory variables, and both sparse and nonsparse models, in the sense that they may include a small and large fraction of the potential variables (and even all). The comparison is done by a comprehensive simulation study, using a quantitative framework for performance comparisons in the form of empirical minimaxity relative to a "random oracle": the oracle model selection performance on data dependent forward selected family of potential models. We show that FDR based procedures have good performance, and in particular the newly proposed method, emerges as having empirical minimax performance. Interestingly, using FDR level of 0.05 is a global best.