FeatureCuts: Feature Selection for Large Data by Optimizing the Cutoff

TL;DR

FeatureCuts is a new feature selection algorithm that adaptively determines the optimal feature cutoff, significantly reducing features and computation time while maintaining model accuracy across various datasets.

Contribution

Introduces FeatureCuts, a novel adaptive feature selection method that improves feature reduction and efficiency over existing techniques, scalable for large datasets.

Findings

Achieves 15% more feature reduction on average

Reduces computation time by up to 99.6%

Enables 25% more feature reduction with wrapper methods

Abstract

In machine learning, the process of feature selection involves finding a reduced subset of features that captures most of the information required to train an accurate and efficient model. This work presents FeatureCuts, a novel feature selection algorithm that adaptively selects the optimal feature cutoff after performing filter ranking. Evaluated on 14 publicly available datasets and one industry dataset, FeatureCuts achieved, on average, 15 percentage points more feature reduction and up to 99.6% less computation time while maintaining model performance, compared to existing state-of-the-art methods. When the selected features are used in a wrapper method such as Particle Swarm Optimization (PSO), it enables 25 percentage points more feature reduction, requires 66% less computation time, and maintains model performance when compared to PSO alone. The minimal overhead of FeatureCuts makes it scalable for large datasets typically seen in enterprise applications.