Adjusted Measures for Feature Selection Stability for Data Sets with Similar Features

TL;DR

This paper introduces new adjusted stability measures for feature selection that effectively handle highly similar or correlated features, addressing limitations of existing measures and improving reliability in feature stability assessment.

Contribution

The authors propose novel adjusted stability measures that account for feature similarities, overcoming theoretical drawbacks of previous methods.

Findings

New stability measure effectively treats highly similar features as exchangeable.

Proposed measures outperform existing ones on artificial and real datasets.

Recommended measure improves feature stability evaluation in correlated feature scenarios.

Abstract

For data sets with similar features, for example highly correlated features, most existing stability measures behave in an undesired way: They consider features that are almost identical but have different identifiers as different features. Existing adjusted stability measures, that is, stability measures that take into account the similarities between features, have major theoretical drawbacks. We introduce new adjusted stability measures that overcome these drawbacks. We compare them to each other and to existing stability measures based on both artificial and real sets of selected features. Based on the results, we suggest using one new stability measure that considers highly similar features as exchangeable.