Influence of the Event Rate on Discrimination Abilities of Bankruptcy Prediction Models

TL;DR

This study examines how varying the event rate impacts the discrimination performance of different bankruptcy prediction models, highlighting the robustness of Bayesian Networks and sensitivity of Support Vector Machines.

Contribution

It systematically analyzes the effect of different event rates on multiple models' discrimination abilities in bankruptcy prediction.

Findings

Bayesian Network is most insensitive to event rate changes.

Support Vector Machine is most sensitive to event rate variations.

Model performance varies significantly with event rate adjustments.

Abstract

In bankruptcy prediction, the proportion of events is very low, which is often oversampled to eliminate this bias. In this paper, we study the influence of the event rate on discrimination abilities of bankruptcy prediction models. First the statistical association and significance of public records and firmographics indicators with the bankruptcy were explored. Then the event rate was oversampled from 0.12% to 10%, 20%, 30%, 40%, and 50%, respectively. Seven models were developed, including Logistic Regression, Decision Tree, Random Forest, Gradient Boosting, Support Vector Machine, Bayesian Network, and Neural Network. Under different event rates, models were comprehensively evaluated and compared based on Kolmogorov-Smirnov Statistic, accuracy, F1 score, Type I error, Type II error, and ROC curve on the hold-out dataset with their best probability cut-offs. Results show that Bayesian Network is the most insensitive to the event rate, while Support Vector Machine is the most sensitive.