Fibonacci and k-Subsecting Recursive Feature Elimination

TL;DR

This paper introduces two new recursive feature elimination algorithms, Fibonacci- and k-Subsecting RFE, which efficiently select smaller feature subsets faster than standard RFE without sacrificing accuracy.

Contribution

The paper presents novel RFE-inspired algorithms that remove features in logarithmic steps, improving speed while maintaining predictive performance.

Findings

Fibonacci and k-Subsecting RFE are faster than standard RFE.

They select smaller feature subsets with comparable accuracy.

Effective on high-dimensional datasets and practical case studies.

Abstract

Feature selection is a data mining task with the potential of speeding up classification algorithms, enhancing model comprehensibility, and improving learning accuracy. However, finding a subset of features that is optimal in terms of predictive accuracy is usually computationally intractable. Out of several heuristic approaches to dealing with this problem, the Recursive Feature Elimination (RFE) algorithm has received considerable interest from data mining practitioners. In this paper, we propose two novel algorithms inspired by RFE, called Fibonacci- and k-Subsecting Recursive Feature Elimination, which remove features in logarithmic steps, probing the wrapped classifier more densely for the more promising feature subsets. The proposed algorithms are experimentally compared against RFE on 28 highly multidimensional datasets and evaluated in a practical case study involving 3D electron density maps from the Protein Data Bank. The results show that Fibonacci and k-Subsecting Recursive Feature Elimination are capable of selecting a smaller subset of features much faster than standard RFE, while achieving comparable predictive performance.