A model-free feature selection technique of feature screening and random forest based recursive feature elimination

TL;DR

This paper introduces a fully nonparametric, model-free feature selection method combining the fused Kolmogorov filter with random forest RFE, effective for ultra-high dimensional data across various applications.

Contribution

It proposes a novel two-phase feature selection approach that reduces computational complexity and overcomes model limitations, with proven consistency and superior performance.

Findings

Method is selection consistent under weak conditions

Demonstrates superior accuracy in simulations

Effective in real-world high-dimensional datasets

Abstract

In this paper, we propose a model-free feature selection method for ultra-high dimensional data with mass features. This is a two phases procedure that we propose to use the fused Kolmogorov filter with the random forest based RFE to remove model limitations and reduce the computational complexity. The method is fully nonparametric and can work with various types of datasets. It has several appealing characteristics, i.e., accuracy, model-free, and computational efficiency, and can be widely used in practical problems, such as multiclass classification, nonparametric regression, and Poisson regression, among others. We show that the proposed method is selection consistent and $L_2$ consistent under weak regularity conditions. We further demonstrate the superior performance of the proposed method over other existing methods by simulations and real data examples.