MVMR-FS : Non-parametric feature selection algorithm based on Maximum inter-class Variation and Minimum Redundancy

TL;DR

This paper introduces MVMR-FS, a non-parametric feature selection algorithm that effectively measures relevance and redundancy in continuous data using kernel density estimation, improving accuracy over existing methods.

Contribution

The paper proposes a novel non-parametric feature selection method based on maximum inter-class variation and minimum redundancy, utilizing kernel density estimation and an adaptive genetic algorithm.

Findings

Achieves highest average accuracy compared to ten state-of-the-art methods.

Improves accuracy by 5% to 11% over existing feature selection techniques.

Effectively measures relevance and redundancy in continuous data.

Abstract

How to accurately measure the relevance and redundancy of features is an age-old challenge in the field of feature selection. However, existing filter-based feature selection methods cannot directly measure redundancy for continuous data. In addition, most methods rely on manually specifying the number of features, which may introduce errors in the absence of expert knowledge. In this paper, we propose a non-parametric feature selection algorithm based on maximum inter-class variation and minimum redundancy, abbreviated as MVMR-FS. We first introduce supervised and unsupervised kernel density estimation on the features to capture their similarities and differences in inter-class and overall distributions. Subsequently, we present the criteria for maximum inter-class variation and minimum redundancy (MVMR), wherein the inter-class probability distributions are employed to reflect feature relevance and the distances between overall probability distributions are used to quantify redundancy. Finally, we employ an AGA to search for the feature subset that minimizes the MVMR. Compared with ten state-of-the-art methods, MVMR-FS achieves the highest average accuracy and improves the accuracy by 5% to 11%.