INSIGHT: Explainable Weakly-Supervised Medical Image Analysis

TL;DR

INSIGHT is a novel weakly-supervised method for medical image analysis that generates heatmaps to localize critical details, achieving state-of-the-art results without post-hoc visualization.

Contribution

It introduces a heatmap-integrated aggregator that captures fine details and suppresses false positives, advancing weakly-supervised medical image analysis.

Findings

State-of-the-art classification accuracy on CT and WSI benchmarks

High performance in weakly-labeled semantic segmentation

Effective localization of diagnostically relevant regions

Abstract

Due to their large sizes, volumetric scans and whole-slide pathology images (WSIs) are often processed by extracting embeddings from local regions and then an aggregator makes predictions from this set. However, current methods require post-hoc visualization techniques (e.g., Grad-CAM) and often fail to localize small yet clinically crucial details. To address these limitations, we introduce INSIGHT, a novel weakly-supervised aggregator that integrates heatmap generation as an inductive bias. Starting from pre-trained feature maps, INSIGHT employs a detection module with small convolutional kernels to capture fine details and a context module with a broader receptive field to suppress local false positives. The resulting internal heatmap highlights diagnostically relevant regions. On CT and WSI benchmarks, INSIGHT achieves state-of-the-art classification results and high weakly-labeled semantic segmentation performance. Project website and code are available at: https://zhangdylan83.github.io/ewsmia/