SegX: Improving Interpretability of Clinical Image Diagnosis with Segmentation-based Enhancement

TL;DR

SegX enhances interpretability of medical image diagnosis by aligning explanation maps with clinically relevant regions using segmentation, and SegU quantifies prediction uncertainty, improving reliability in clinical AI applications.

Contribution

Introduces SegX and SegU, novel segmentation-based methods that improve interpretability and uncertainty quantification in medical image analysis.

Findings

SegX consistently improves interpretability across datasets.

SegU reliably reflects the correctness of predictions.

Approach is model-agnostic and enhances clinical decision-making.

Abstract

Deep learning-based medical image analysis faces a significant barrier due to the lack of interpretability. Conventional explainable AI (XAI) techniques, such as Grad-CAM and SHAP, often highlight regions outside clinical interests. To address this issue, we propose Segmentation-based Explanation (SegX), a plug-and-play approach that enhances interpretability by aligning the model's explanation map with clinically relevant areas leveraging the power of segmentation models. Furthermore, we introduce Segmentation-based Uncertainty Assessment (SegU), a method to quantify the uncertainty of the prediction model by measuring the 'distance' between interpretation maps and clinically significant regions. Our experiments on dermoscopic and chest X-ray datasets show that SegX improves interpretability consistently across mortalities, and the certainty score provided by SegU reliably reflects the correctness of the model's predictions. Our approach offers a model-agnostic enhancement to medical image diagnosis towards reliable and interpretable AI in clinical decision-making.