ManiFeSt: Manifold-based Feature Selection for Small Data Sets

TL;DR

ManiFeSt introduces a manifold-based feature selection method tailored for small datasets, leveraging Riemannian geometry and spectral analysis to improve feature relevance and classification accuracy.

Contribution

The paper presents a novel multivariate feature selection approach that models feature manifolds and captures their differences using Riemannian geometry, enhancing performance on small data sets.

Findings

Outperforms existing methods in feature selection accuracy

Improves classification and generalization on benchmark datasets

Effectively captures multi-feature associations to avoid overfitting

Abstract

In this paper, we present a new method for few-sample supervised feature selection (FS). Our method first learns the manifold of the feature space of each class using kernels capturing multi-feature associations. Then, based on Riemannian geometry, a composite kernel is computed, extracting the differences between the learned feature associations. Finally, a FS score based on spectral analysis is proposed. Considering multi-feature associations makes our method multivariate by design. This in turn allows for the extraction of the hidden manifold underlying the features and avoids overfitting, facilitating few-sample FS. We showcase the efficacy of our method on illustrative examples and several benchmarks, where our method demonstrates higher accuracy in selecting the informative features compared to competing methods. In addition, we show that our FS leads to improved classification and better generalization when applied to test data.