MediCLIP: Adapting CLIP for Few-shot Medical Image Anomaly Detection

TL;DR

MediCLIP adapts the CLIP model for few-shot medical image anomaly detection by using self-supervised fine-tuning and synthetic anomaly tasks, achieving state-of-the-art results with limited data.

Contribution

The paper introduces MediCLIP, a novel method that fine-tunes CLIP for few-shot medical anomaly detection using synthetic disease pattern generation.

Findings

MediCLIP outperforms existing methods in anomaly detection accuracy.

Synthetic anomaly tasks effectively transfer CLIP's generalization to medical imaging.

The approach requires only a few normal images for effective detection.

Abstract

In the field of medical decision-making, precise anomaly detection in medical imaging plays a pivotal role in aiding clinicians. However, previous work is reliant on large-scale datasets for training anomaly detection models, which increases the development cost. This paper first focuses on the task of medical image anomaly detection in the few-shot setting, which is critically significant for the medical field where data collection and annotation are both very expensive. We propose an innovative approach, MediCLIP, which adapts the CLIP model to few-shot medical image anomaly detection through self-supervised fine-tuning. Although CLIP, as a vision-language model, demonstrates outstanding zero-/fewshot performance on various downstream tasks, it still falls short in the anomaly detection of medical images. To address this, we design a series of medical image anomaly synthesis tasks to simulate common disease patterns in medical imaging, transferring the powerful generalization capabilities of CLIP to the task of medical image anomaly detection. When only few-shot normal medical images are provided, MediCLIP achieves state-of-the-art performance in anomaly detection and location compared to other methods. Extensive experiments on three distinct medical anomaly detection tasks have demonstrated the superiority of our approach. The code is available at https://github.com/cnulab/MediCLIP.