The Impact of Pseudo-Science in Financial Loans Risk Prediction

TL;DR

This paper examines how pseudo-scientific assumptions and survival bias affect machine learning models in financial risk prediction, revealing that models may appear to improve while actually becoming more unfair and biased.

Contribution

It highlights the societal impact of pseudo-science and survival bias in financial loan risk models, showing that accuracy improvements can mask increasing unfairness.

Findings

Models with survival bias show slight accuracy deterioration over time.

Recall and precision improve over time despite bias.

Socially optimal models may not significantly reduce accuracy.

Abstract

We study the societal impact of pseudo-scientific assumptions for predicting the behavior of people in a straightforward application of machine learning to risk prediction in financial lending. This use case also exemplifies the impact of survival bias in loan return prediction. We analyze the models in terms of their accuracy and social cost, showing that the socially optimal model may not imply a significant accuracy loss for this downstream task. Our results are verified for commonly used learning methods and datasets. Our findings also show that there is a natural dynamic when training models that suffer survival bias where accuracy slightly deteriorates, and whose recall and precision improves with time. These results act as an illusion, leading the observer to believe that the system is getting better, when in fact the model is suffering from increasingly more unfairness and survival bias.