Studying the Effects of Self-Attention for Medical Image Analysis

TL;DR

This paper compares various self-attention mechanisms in medical image analysis, demonstrating their potential to improve focus on clinically relevant regions and enhance robustness in computer vision tasks.

Contribution

It provides a comprehensive comparison of state-of-the-art self-attention methods in medical imaging, including quantitative, qualitative, and user-centric evaluations.

Findings

Self-attention improves focus on important clinical features.

Different attention mechanisms have varying impacts on performance.

User studies highlight clinical relevance of attention-based models.

Abstract

When the trained physician interprets medical images, they understand the clinical importance of visual features. By applying cognitive attention, they apply greater focus onto clinically relevant regions while disregarding unnecessary features. The use of computer vision to automate the classification of medical images is widely studied. However, the standard convolutional neural network (CNN) does not necessarily employ subconscious feature relevancy evaluation techniques similar to the trained medical specialist and evaluates features more generally. Self-attention mechanisms enable CNNs to focus more on semantically important regions or aggregated relevant context with long-range dependencies. By using attention, medical image analysis systems can potentially become more robust by focusing on more important clinical feature regions. In this paper, we provide a comprehensive comparison of various state-of-the-art self-attention mechanisms across multiple medical image analysis tasks. Through both quantitative and qualitative evaluations along with a clinical user-centric survey study, we aim to provide a deeper understanding of the effects of self-attention in medical computer vision tasks.