Provable FDR Control for Deep Feature Selection: Deep MLPs and Beyond

TL;DR

This paper introduces a deep neural network-based feature selection method that provably controls the false discovery rate (FDR) across various architectures, with theoretical guarantees and empirical validation.

Contribution

It provides the first theoretical FDR control guarantee for deep learning-based feature selection applicable to diverse architectures.

Findings

Supports FDR control in deep networks with asymptotic normality of feature importance

Applicable to various architectures including MLPs, CNNs, RNNs, and attention mechanisms

Numerical experiments confirm theoretical results

Abstract

We develop a flexible feature selection framework based on deep neural networks that approximately controls the false discovery rate (FDR), a measure of Type-I error. The method applies to architectures whose first layer is fully connected. From the second layer onward, it accommodates multilayer perceptrons (MLPs) of arbitrary width and depth, convolutional and recurrent networks, attention mechanisms, residual connections, and dropout. The procedure also accommodates stochastic gradient descent with data-independent initializations and learning rates. To the best of our knowledge, this is the first work to provide a theoretical guarantee of FDR control for feature selection within such a general deep learning setting. Our analysis is built upon a multi-index data-generating model and an asymptotic regime in which the feature dimension $n$ diverges faster than the latent dimension $q^{*}$, while the sample size, the number of training iterations, the network depth, and hidden layer widths are left unrestricted. Under this setting, we show that each coordinate of the gradient-based feature-importance vector admits a marginal normal approximation, thereby supporting the validity of asymptotic FDR control. As a theoretical limitation, we assume $\mathbf{B}$-right orthogonal invariance of the design matrix, and we discuss broader generalizations. We also present numerical experiments that underscore the theoretical findings.