Error-based Knockoffs Inference for Controlled Feature Selection

TL;DR

This paper introduces an error-based knockoff inference method that improves feature selection by providing flexible, model-free control over false discovery metrics in high-dimensional data.

Contribution

It proposes a novel, error-based approach integrating knockoff features and stepdown procedures, offering theoretical guarantees without relying on specific regression models.

Findings

Effective control of FDR, FDP, and k-FWER demonstrated

Competitive performance on simulated and real datasets

Flexible, model-free feature selection method

Abstract

Recently, the scheme of model-X knockoffs was proposed as a promising solution to address controlled feature selection under high-dimensional finite-sample settings. However, the procedure of model-X knockoffs depends heavily on the coefficient-based feature importance and only concerns the control of false discovery rate (FDR). To further improve its adaptivity and flexibility, in this paper, we propose an error-based knockoff inference method by integrating the knockoff features, the error-based feature importance statistics, and the stepdown procedure together. The proposed inference procedure does not require specifying a regression model and can handle feature selection with theoretical guarantees on controlling false discovery proportion (FDP), FDR, or k-familywise error rate (k-FWER). Empirical evaluations demonstrate the competitive performance of our approach on both simulated and real data.