Deep Learning Under the Microscope: Improving the Interpretability of Medical Imaging Neural Networks

TL;DR

This paper introduces a novel interpretability method for histological WSI classification using a DNN with MIL, providing detailed heatmaps that improve understanding of model decisions, validated on challenging datasets and expert feedback.

Contribution

A new interpretation approach for histology neural networks using heatmaps and MIL, enhancing explainability without guiding attention, validated on real datasets with expert validation.

Findings

Outperforms baseline interpretability methods on histology datasets

Provides detailed heatmaps that clarify model decision processes

Expert pathologists recognize potential clinical utility

Abstract

In this paper, we propose a novel interpretation method tailored to histological Whole Slide Image (WSI) processing. A Deep Neural Network (DNN), inspired by Bag-of-Features models is equipped with a Multiple Instance Learning (MIL) branch and trained with weak supervision for WSI classification. MIL avoids label ambiguity and enhances our model's expressive power without guiding its attention. We utilize a fine-grained logit heatmap of the models activations to interpret its decision-making process. The proposed method is quantitatively and qualitatively evaluated on two challenging histology datasets, outperforming a variety of baselines. In addition, two expert pathologists were consulted regarding the interpretability provided by our method and acknowledged its potential for integration into several clinical applications.