Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectives

TL;DR

This paper reviews the application of Transformer models in medical imaging, comparing their properties with CNNs, summarizing current research progress across various tasks, and discussing future directions in the field.

Contribution

It provides a comprehensive, property-based organization of Transformer approaches in medical imaging, highlighting distinctions from CNNs and offering insights into future research avenues.

Findings

Transformers are increasingly applied in medical image segmentation, recognition, and detection.

Key properties of Transformers differentiate them from CNNs, influencing their design and application.

The review identifies current challenges and potential future developments in Transformer-based medical imaging.

Abstract

Transformer, the latest technological advance of deep learning, has gained prevalence in natural language processing or computer vision. Since medical imaging bear some resemblance to computer vision, it is natural to inquire about the status quo of Transformers in medical imaging and ask the question: can the Transformer models transform medical imaging? In this paper, we attempt to make a response to the inquiry. After a brief introduction of the fundamentals of Transformers, especially in comparison with convolutional neural networks (CNNs), and highlighting key defining properties that characterize the Transformers, we offer a comprehensive review of the state-of-the-art Transformer-based approaches for medical imaging and exhibit current research progresses made in the areas of medical image segmentation, recognition, detection, registration, reconstruction, enhancement, etc. In particular, what distinguishes our review lies in its organization based on the Transformer's key defining properties, which are mostly derived from comparing the Transformer and CNN, and its type of architecture, which specifies the manner in which the Transformer and CNN are combined, all helping the readers to best understand the rationale behind the reviewed approaches. We conclude with discussions of future perspectives.