Graph-based Extreme Feature Selection for Multi-class Classification Tasks

TL;DR

This paper introduces a graph-based filter feature selection method for multi-class classification that reduces features by combining Jeffries-Matusita distance with diffusion maps, improving interpretability and performance.

Contribution

It proposes a novel graph-based feature selection algorithm using Jeffries-Matusita distance and diffusion maps for multi-class datasets, enabling effective feature reduction and visualization.

Findings

Significant reduction in feature count while maintaining classification accuracy.

Enhanced visualization of feature space through low-dimensional embedding.

Outperforms existing filter methods on public datasets.

Abstract

When processing high-dimensional datasets, a common pre-processing step is feature selection. Filter-based feature selection algorithms are not tailored to a specific classification method, but rather rank the relevance of each feature with respect to the target and the task. This work focuses on a graph-based, filter feature selection method that is suited for multi-class classifications tasks. We aim to drastically reduce the number of selected features, in order to create a sketch of the original data that codes valuable information for the classification task. The proposed graph-based algorithm is constructed by combing the Jeffries-Matusita distance with a non-linear dimension reduction method, diffusion maps. Feature elimination is performed based on the distribution of the features in the low-dimensional space. Then, a very small number of feature that have complementary separation strengths, are selected. Moreover, the low-dimensional embedding allows to visualize the feature space. Experimental results are provided for public datasets and compared with known filter-based feature selection techniques.