Deep Learning vs. Gradient Boosting: Benchmarking state-of-the-art machine learning algorithms for credit scoring

TL;DR

This study benchmarks deep learning and gradient boosting algorithms for credit scoring, finding that gradient boosting generally outperforms deep learning in accuracy and speed across various datasets, making it the preferred choice.

Contribution

The paper provides a comprehensive comparison of DL and GBM for credit scoring, highlighting the circumstances under which each performs best and establishing GBM as the more practical, efficient option.

Findings

GBM generally outperforms DL in accuracy and speed

GBM has lower computational requirements and faster training times

Model choice depends on dataset characteristics

Abstract

Artificial intelligence (AI) and machine learning (ML) have become vital to remain competitive for financial services companies around the globe. The two models currently competing for the pole position in credit risk management are deep learning (DL) and gradient boosting machines (GBM). This paper benchmarked those two algorithms in the context of credit scoring using three distinct datasets with different features to account for the reality that model choice/power is often dependent on the underlying characteristics of the dataset. The experiment has shown that GBM tends to be more powerful than DL and has also the advantage of speed due to lower computational requirements. This makes GBM the winner and choice for credit scoring. However, it was also shown that the outperformance of GBM is not always guaranteed and ultimately the concrete problem scenario or dataset will determine the final model choice. Overall, based on this study both algorithms can be considered state-of-the-art for binary classification tasks on structured datasets, while GBM should be the go-to solution for most problem scenarios due to easier use, significantly faster training time, and superior accuracy.