Is visual explanation with Grad-CAM more reliable for deeper neural networks? a case study with automatic pneumothorax diagnosis

TL;DR

This study examines how the reliability of Grad-CAM visual explanations varies across different deep neural network architectures in the context of automatic pneumothorax diagnosis from X-ray images.

Contribution

It provides a systematic analysis of Grad-CAM's robustness and effectiveness across various deep learning models with different depths and architectures.

Findings

Deeper networks do not always improve diagnosis accuracy.

Grad-CAM effectiveness varies among different architectures.

Deeper networks do not necessarily enhance interpretability.

Abstract

While deep learning techniques have provided the state-of-the-art performance in various clinical tasks, explainability regarding their decision-making process can greatly enhance the credence of these methods for safer and quicker clinical adoption. With high flexibility, Gradient-weighted Class Activation Mapping (Grad-CAM) has been widely adopted to offer intuitive visual interpretation of various deep learning models' reasoning processes in computer-assisted diagnosis. However, despite the popularity of the technique, there is still a lack of systematic study on Grad-CAM's performance on different deep learning architectures. In this study, we investigate its robustness and effectiveness across different popular deep learning models, with a focus on the impact of the networks' depths and architecture types, by using a case study of automatic pneumothorax diagnosis in X-ray scans. Our results show that deeper neural networks do not necessarily contribute to a strong improvement of pneumothorax diagnosis accuracy, and the effectiveness of GradCAM also varies among different network architectures.