Mask of truth: model sensitivity to unexpected regions of medical images

TL;DR

This study investigates how CNN models for medical image classification rely on non-relevant regions and spurious correlations, revealing their robustness and interpretability issues through masking experiments and explainability tools.

Contribution

It demonstrates that CNNs can perform well even when clinically relevant regions are masked, highlighting reliance on spurious cues and the need for better validation methods.

Findings

Models trained on full images perform well even without the ROI.

Spurious correlations are present in the Chaksu dataset.

Explainability analysis reveals reliance on non-relevant regions.

Abstract

The development of larger models for medical image analysis has led to increased performance. However, it also affected our ability to explain and validate model decisions. Models can use non-relevant parts of images, also called spurious correlations or shortcuts, to obtain high performance on benchmark datasets but fail in real-world scenarios. In this work, we challenge the capacity of convolutional neural networks (CNN) to classify chest X-rays and eye fundus images while masking out clinically relevant parts of the image. We show that all models trained on the PadChest dataset, irrespective of the masking strategy, are able to obtain an Area Under the Curve (AUC) above random. Moreover, the models trained on full images obtain good performance on images without the region of interest (ROI), even superior to the one obtained on images only containing the ROI. We also reveal a possible spurious correlation in the Chaksu dataset while the performances are more aligned with the expectation of an unbiased model. We go beyond the performance analysis with the usage of the explainability method SHAP and the analysis of embeddings. We asked a radiology resident to interpret chest X-rays under different masking to complement our findings with clinical knowledge. Our code is available at https://github.com/TheoSourget/MMC_Masking and https://github.com/TheoSourget/MMC_Masking_EyeFundus