Explainable bank failure prediction models: Counterfactual explanations to reduce the failure risk

TL;DR

This paper evaluates counterfactual explanation methods to improve interpretability of complex bank failure prediction models, emphasizing the importance of validity, proximity, and sparsity in explanations.

Contribution

It compares several counterfactual generation techniques and data balancing strategies, identifying the most effective methods for explaining black box models in banking risk prediction.

Findings

Nearest Instance Counterfactual Explanation with cost sensitive approach yields high-quality explanations.

Multi Objective and Nearest Instance methods outperform others in validity, proximity, and sparsity.

Cost sensitive approach provides the most desirable counterfactual explanations.

Abstract

The accuracy and understandability of bank failure prediction models are crucial. While interpretable models like logistic regression are favored for their explainability, complex models such as random forest, support vector machines, and deep learning offer higher predictive performance but lower explainability. These models, known as black boxes, make it difficult to derive actionable insights. To address this challenge, using counterfactual explanations is suggested. These explanations demonstrate how changes in input variables can alter the model output and suggest ways to mitigate bank failure risk. The key challenge lies in selecting the most effective method for generating useful counterfactuals, which should demonstrate validity, proximity, sparsity, and plausibility. The paper evaluates several counterfactual generation methods: WhatIf, Multi Objective, and Nearest Instance Counterfactual Explanation, and also explores resampling methods like undersampling, oversampling, SMOTE, and the cost sensitive approach to address data imbalance in bank failure prediction in the US. The results indicate that the Nearest Instance Counterfactual Explanation method yields higher quality counterfactual explanations, mainly using the cost sensitive approach. Overall, the Multi Objective Counterfactual and Nearest Instance Counterfactual Explanation methods outperform others regarding validity, proximity, and sparsity metrics, with the cost sensitive approach providing the most desirable counterfactual explanations. These findings highlight the variability in the performance of counterfactual generation methods across different balancing strategies and machine learning models, offering valuable strategies to enhance the utility of black box bank failure prediction models.