The effect of different feature selection methods on models created with XGBoost

TL;DR

This paper investigates how various feature selection techniques impact XGBoost models, finding that feature reduction methods do not significantly affect accuracy but may reduce computational costs.

Contribution

It demonstrates that feature selection methods do not significantly change XGBoost accuracy, challenging traditional assumptions about noise removal and overfitting.

Findings

Feature selection methods do not significantly alter prediction accuracy.

Dimensionality reduction can lower computational complexity.

Traditional noise removal may not be necessary for XGBoost.

Abstract

This study examines the effect that different feature selection methods have on models created with XGBoost, a popular machine learning algorithm with superb regularization methods. It shows that three different ways for reducing the dimensionality of features produces no statistically significant change in the prediction accuracy of the model. This suggests that the traditional idea of removing the noisy training data to make sure models do not overfit may not apply to XGBoost. But it may still be viable in order to reduce computational complexity.