Evaluating AI fairness in credit scoring with the BRIO tool

TL;DR

This paper introduces BRIO, a comprehensive tool for analyzing fairness and ethical issues in AI credit scoring models, using the German Credit Dataset to identify biases across demographic groups.

Contribution

It extends the BRIO tool with a full unfairness risk evaluation module and demonstrates its application in credit scoring fairness analysis.

Findings

Quantified biases across demographic groups in credit scoring.

Identified potential sources of discrimination in the model.

Provided insights for ethical and fair AI credit scoring practices.

Abstract

We present a method for quantitative, in-depth analyses of fairness issues in AI systems with an application to credit scoring. To this aim we use BRIO, a tool for the evaluation of AI systems with respect to social unfairness and, more in general, ethically undesirable behaviours. It features a model-agnostic bias detection module, presented in \cite{DBLP:conf/beware/CoragliaDGGPPQ23}, to which a full-fledged unfairness risk evaluation module is added. As a case study, we focus on the context of credit scoring, analysing the UCI German Credit Dataset \cite{misc_statlog_(german_credit_data)_144}. We apply the BRIO fairness metrics to several, socially sensitive attributes featured in the German Credit Dataset, quantifying fairness across various demographic segments, with the aim of identifying potential sources of bias and discrimination in a credit scoring model. We conclude by combining our results with a revenue analysis.