CLIP in Medical Imaging: A Survey
Zihao Zhao, Yuxiao Liu, Han Wu, Mei Wang, Yonghao Li, Sheng Wang, Lin, Teng, Disheng Liu, Zhiming Cui, Qian Wang, Dinggang Shen

TL;DR
This survey comprehensively reviews the application of CLIP in medical imaging, covering its adaptation, practical uses, limitations, and future directions to enhance understanding and development in this domain.
Contribution
It provides an in-depth analysis of how CLIP pre-training and applications are adapted for medical imaging, highlighting challenges and future research directions.
Findings
CLIP shows promise in medical image classification and diagnosis.
Adaptation strategies improve CLIP performance on medical data.
Identifies limitations and proposes future research directions.
Abstract
Contrastive Language-Image Pre-training (CLIP), a simple yet effective pre-training paradigm, successfully introduces text supervision to vision models. It has shown promising results across various tasks due to its generalizability and interpretability. The use of CLIP has recently gained increasing interest in the medical imaging domain, serving as a pre-training paradigm for image-text alignment, or a critical component in diverse clinical tasks. With the aim of facilitating a deeper understanding of this promising direction, this survey offers an in-depth exploration of the CLIP within the domain of medical imaging, regarding both refined CLIP pre-training and CLIP-driven applications. In this paper, we (1) first start with a brief introduction to the fundamentals of CLIP methodology; (2) then investigate the adaptation of CLIP pre-training in the medical imaging domain, focusing on…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsRadiomics and Machine Learning in Medical Imaging · Artificial Intelligence in Healthcare and Education · AI in cancer detection
MethodsContrastive Language-Image Pre-training
