UbiQVision: Quantifying Uncertainty in XAI for Image Recognition

TL;DR

This paper introduces a framework that quantifies uncertainty in SHAP explanations for medical image recognition, improving interpretability amidst model complexity and data variability.

Contribution

It proposes a novel approach combining Dirichlet sampling and Dempster-Shafer theory to assess the reliability of SHAP explanations in medical imaging.

Findings

Framework effectively quantifies uncertainty in SHAP explanations.

Evaluated on diverse medical datasets with varying noise and modalities.

Enhances trustworthiness of model interpretability in critical applications.

Abstract

Recent advances in deep learning have led to its widespread adoption across diverse domains, including medical imaging. This progress is driven by increasingly sophisticated model architectures, such as ResNets, Vision Transformers, and Hybrid Convolutional Neural Networks, that offer enhanced performance at the cost of greater complexity. This complexity often compromises model explainability and interpretability. SHAP has emerged as a prominent method for providing interpretable visualizations that aid domain experts in understanding model predictions. However, SHAP explanations can be unstable and unreliable in the presence of epistemic and aleatoric uncertainty. In this study, we address this challenge by using Dirichlet posterior sampling and Dempster-Shafer theory to quantify the uncertainty that arises from these unstable explanations in medical imaging applications. The framework uses a belief, plausible, and fusion map approach alongside statistical quantitative analysis to produce quantification of uncertainty in SHAP. Furthermore, we evaluated our framework on three medical imaging datasets with varying class distributions, image qualities, and modality types which introduces noise due to varying image resolutions and modality-specific aspect covering the examples from pathology, ophthalmology, and radiology, introducing significant epistemic uncertainty.