Transformers in Medical Image Analysis: A Review

TL;DR

This review paper discusses the application of Transformer models in medical image analysis, covering their core concepts, architectures, challenges, and potential for clinical applications.

Contribution

It provides a comprehensive overview of Transformer architectures tailored for medical imaging and discusses key challenges and limitations in their current use.

Findings

Transformers have been successfully applied to various medical imaging tasks.

Key challenges include model efficiency and integration with other techniques.

The review highlights future directions for Transformer research in medicine.

Abstract

Transformers have dominated the field of natural language processing, and recently impacted the computer vision area. In the field of medical image analysis, Transformers have also been successfully applied to full-stack clinical applications, including image synthesis/reconstruction, registration, segmentation, detection, and diagnosis. Our paper aims to promote awareness and application of Transformers in the field of medical image analysis. Specifically, we first overview the core concepts of the attention mechanism built into Transformers and other basic components. Second, we review various Transformer architectures tailored for medical image applications and discuss their limitations. Within this review, we investigate key challenges revolving around the use of Transformers in different learning paradigms, improving the model efficiency, and their coupling with other techniques. We hope this review can give a comprehensive picture of Transformers to the readers in the field of medical image analysis.