FSDEM: Feature Selection Dynamic Evaluation Metric

TL;DR

This paper introduces FSDEM, a new dynamic evaluation metric for feature selection algorithms that assesses both their performance and stability, providing a more comprehensive evaluation method.

Contribution

The paper presents a novel, flexible evaluation metric for feature selection algorithms that addresses limitations of previous metrics and enables more reliable assessments.

Findings

The proposed metric effectively evaluates feature selection performance.

The metric assesses stability alongside performance.

Empirical results validate the metric's usefulness.

Abstract

Expressive evaluation metrics are indispensable for informative experiments in all areas, and while several metrics are established in some areas, in others, such as feature selection, only indirect or otherwise limited evaluation metrics are found. In this paper, we propose a novel evaluation metric to address several problems of its predecessors and allow for flexible and reliable evaluation of feature selection algorithms. The proposed metric is a dynamic metric with two properties that can be used to evaluate both the performance and the stability of a feature selection algorithm. We conduct several empirical experiments to illustrate the use of the proposed metric in the successful evaluation of feature selection algorithms. We also provide a comparison and analysis to show the different aspects involved in the evaluation of the feature selection algorithms. The results indicate that the proposed metric is successful in carrying out the evaluation task for feature selection algorithms. This paper is an extended version of a paper published at SISAP 2024.