Embedded all relevant feature selection with Random Ferns

TL;DR

This paper introduces a novel method for all relevant feature selection integrated into the training process of random ferns, aiming to improve feature relevance detection without excessive computational costs.

Contribution

The paper proposes a new embedded feature selection method using shadow attributes within the random ferns classifier, enhancing relevance detection during training.

Findings

Effective in identifying relevant features

Limited to small dimensions or combined with other methods

Shows promise but with some limitations

Abstract

Many machine learning methods can produce variable importance scores expressing the usability of each feature in context of the produced model; those scores on their own are yet not sufficient to generate feature selection, especially when an all relevant selection is required. Although there are wrapper methods aiming to solve this problem, they introduce a substantial increase in the required computational effort. In this paper I investigate an idea of incorporating all relevant selection within the training process by producing importance for implicitly generated shadows, attributes irrelevant by design. I propose and evaluate such a method in context of random ferns classifier. Experiment results confirm the effectiveness of such approach, although show that fully stochastic nature of random ferns limits its applicability either to small dimensions or as a part of a broader feature selection procedure.