Are screening methods useful in feature selection? An empirical study

TL;DR

This study empirically evaluates the effectiveness of filter screening methods in feature selection across various datasets, comparing their impact on different learning algorithms and accuracy metrics.

Contribution

It provides an objective comparison of popular screening methods with multiple learners on real datasets, assessing their usefulness in improving predictive performance.

Findings

Screening methods improved the best learner's prediction on 4 out of 10 datasets.

They were particularly beneficial for certain regression and classification tasks.

Overall, screening methods showed limited but notable improvements in predictive accuracy.

Abstract

Filter or screening methods are often used as a preprocessing step for reducing the number of variables used by a learning algorithm in obtaining a classification or regression model. While there are many such filter methods, there is a need for an objective evaluation of these methods. Such an evaluation is needed to compare them with each other and also to answer whether they are at all useful, or a learning algorithm could do a better job without them. For this purpose, many popular screening methods are partnered in this paper with three regression learners and five classification learners and evaluated on ten real datasets to obtain accuracy criteria such as R-square and area under the ROC curve (AUC). The obtained results are compared through curve plots and comparison tables in order to find out whether screening methods help improve the performance of learning algorithms and how they fare with each other. Our findings revealed that the screening methods were useful in improving the prediction of the best learner on two regression and two classification datasets out of the ten datasets evaluated.