A Sneak Attack on Segmentation of Medical Images Using Deep Neural Network Classifiers

TL;DR

This paper explores using CNN classifiers and Grad-CAM heatmaps for medical image segmentation, revealing potential and limitations of this approach compared to traditional segmentation models.

Contribution

It introduces a novel method of using CNN classifiers and heatmaps for segmentation and proposes an evaluation technique based on retraining classifiers with heatmap-filtered images.

Findings

Heatmaps can locate and segment partial tumor areas.

Using heatmaps alone may not be optimal for segmentation.

CNN classifiers rely mainly on tumor regions for predictions.

Abstract

Instead of using current deep-learning segmentation models (like the UNet and variants), we approach the segmentation problem using trained Convolutional Neural Network (CNN) classifiers, which automatically extract important features from images for classification. Those extracted features can be visualized and formed into heatmaps using Gradient-weighted Class Activation Mapping (Grad-CAM). This study tested whether the heatmaps could be used to segment the classified targets. We also proposed an evaluation method for the heatmaps; that is, to re-train the CNN classifier using images filtered by heatmaps and examine its performance. We used the mean-Dice coefficient to evaluate segmentation results. Results from our experiments show that heatmaps can locate and segment partial tumor areas. But use of only the heatmaps from CNN classifiers may not be an optimal approach for segmentation. We have verified that the predictions of CNN classifiers mainly depend on tumor areas, and dark regions in Grad-CAM's heatmaps also contribute to classification.