The Impact of Machine Learning Uncertainty on the Robustness of Counterfactual Explanations

TL;DR

This paper examines how uncertainty in machine learning models affects the stability and validity of counterfactual explanations, highlighting their sensitivity to model inaccuracies and the importance of incorporating uncertainty awareness.

Contribution

It provides an empirical analysis of counterfactual explanation robustness under model and data uncertainty, emphasizing the need for uncertainty-aware methods.

Findings

Counterfactual explanations are highly sensitive to model uncertainty.

Small decreases in model accuracy can cause large variations in explanations.

Uncertainty-aware explanation methods are necessary for real-world applications.

Abstract

Counterfactual explanations are widely used to interpret machine learning predictions by identifying minimal changes to input features that would alter a model's decision. However, most existing counterfactual methods have not been tested when model and data uncertainty change, resulting in explanations that may be unstable or invalid under real-world variability. In this work, we investigate the robustness of common combinations of machine learning models and counterfactual generation algorithms in the presence of both aleatoric and epistemic uncertainty. Through experiments on synthetic and real-world tabular datasets, we show that counterfactual explanations are highly sensitive to model uncertainty. In particular, we find that even small reductions in model accuracy - caused by increased noise or limited data - can lead to large variations in the generated counterfactuals on average and on individual instances. These findings underscore the need for uncertainty-aware explanation methods in domains such as finance and the social sciences.