Leveraging CAM Algorithms for Explaining Medical Semantic Segmentation

TL;DR

This paper introduces Seg-HiRes-Grad CAM, a novel method that adapts classification-based CAM algorithms for more detailed and consistent explanation of CNN decisions in medical image segmentation tasks.

Contribution

It proposes a transfer approach from classification- to segmentation-based CAM methods, enhancing explainability in medical segmentation.

Findings

Improved saliency pixel identification in segmentation tasks.

Enhanced explainability for medical image analysis.

Outperforms previous segmentation CAM methods.

Abstract

Convolutional neural networks (CNNs) achieve prevailing results in segmentation tasks nowadays and represent the state-of-the-art for image-based analysis. However, the understanding of the accurate decision-making process of a CNN is rather unknown. The research area of explainable artificial intelligence (xAI) primarily revolves around understanding and interpreting this black-box behavior. One way of interpreting a CNN is the use of class activation maps (CAMs) that represent heatmaps to indicate the importance of image areas for the prediction of the CNN. For classification tasks, a variety of CAM algorithms exist. But for segmentation tasks, only one CAM algorithm for the interpretation of the output of a CNN exist. We propose a transfer between existing classification- and segmentation-based methods for more detailed, explainable, and consistent results which show salient pixels in semantic segmentation tasks. The resulting Seg-HiRes-Grad CAM is an extension of the segmentation-based Seg-Grad CAM with the transfer to the classification-based HiRes CAM. Our method improves the previously-mentioned existing segmentation-based method by adjusting it to recently published classification-based methods. Especially for medical image segmentation, this transfer solves existing explainability disadvantages.