Explaining Digital Pathology Models via Clustering Activations

TL;DR

This paper introduces a clustering-based explainability method for digital pathology CNN models that reveals global model behavior and enhances interpretability compared to traditional saliency map techniques.

Contribution

The paper proposes a novel clustering approach for explaining CNN models in digital pathology, providing both global insights and fine-grained details.

Findings

Effective visualization of model clusters

Improved understanding of model decision-making

Validated on prostate cancer detection model

Abstract

We present a clustering-based explainability technique for digital pathology models based on convolutional neural networks. Unlike commonly used methods based on saliency maps, such as occlusion, GradCAM, or relevance propagation, which highlight regions that contribute the most to the prediction for a single slide, our method shows the global behaviour of the model under consideration, while also providing more fine-grained information. The result clusters can be visualised not only to understand the model, but also to increase confidence in its operation, leading to faster adoption in clinical practice. We also evaluate the performance of our technique on an existing model for detecting prostate cancer, demonstrating its usefulness.