Conformal Recursive Feature Elimination

TL;DR

This paper introduces Conformal Recursive Feature Elimination (CRFE), a novel feature selection method leveraging conformal prediction to identify non-redundant features with an automatic stopping criterion, outperforming classical RFE in many cases.

Contribution

The paper presents CRFE, a new conformal prediction-based feature selection method with an automatic stopping rule, improving feature subset quality without relying on classification performance.

Findings

CRFE outperforms RFE on half of the datasets.

CRFE achieves similar performance to RFE on remaining datasets.

CRFE provides effective, non-redundant feature subsets without classification metrics.

Abstract

Unlike traditional statistical methods, Conformal Prediction (CP) allows for the determination of valid and accurate confidence levels associated with individual predictions based only on exchangeability of the data. We here introduce a new feature selection method that takes advantage of the CP framework. Our proposal, named Conformal Recursive Feature Elimination (CRFE), identifies and recursively removes features that increase the non-conformity of a dataset. We also present an automatic stopping criterion for CRFE, as well as a new index to measure consistency between subsets of features. CRFE selections are compared to the classical Recursive Feature Elimination (RFE) method on several multiclass datasets by using multiple partitions of the data. The results show that CRFE clearly outperforms RFE in half of the datasets, while achieving similar performance in the rest. The automatic stopping criterion provides subsets of effective and non-redundant features without computing any classification performance.