Graph-Based Automatic Feature Selection for Multi-Class Classification via Mean Simplified Silhouette

TL;DR

This paper presents a graph-based filter method for automatic feature selection in multi-class classification that efficiently identifies minimal feature subsets without user parameters, maintaining accuracy and reducing computational time.

Contribution

It introduces a novel feature selection approach using the Mean Simplified Silhouette index combined with JM distance and t-SNE, which outperforms existing methods without requiring parameter tuning.

Findings

Achieves comparable accuracy with only 7-30% of features.

Reduces classification time by 15-70%.

Outperforms other filter-based and automatic feature selection methods.

Abstract

This paper introduces a novel graph-based filter method for automatic feature selection (abbreviated as GB-AFS) for multi-class classification tasks. The method determines the minimum combination of features required to sustain prediction performance while maintaining complementary discriminating abilities between different classes. It does not require any user-defined parameters such as the number of features to select. The methodology employs the Jeffries-Matusita (JM) distance in conjunction with t-distributed Stochastic Neighbor Embedding (t-SNE) to generate a low-dimensional space reflecting how effectively each feature can differentiate between each pair of classes. The minimum number of features is selected using our newly developed Mean Simplified Silhouette (abbreviated as MSS) index, designed to evaluate the clustering results for the feature selection task. Experimental results on public data sets demonstrate the superior performance of the proposed GB-AFS over other filter-based techniques and automatic feature selection approaches. Moreover, the proposed algorithm maintained the accuracy achieved when utilizing all features, while using only $7\%$ to $30\%$ of the features. Consequently, this resulted in a reduction of the time needed for classifications, from $15\%$ to $70\%$.