Dual Distillation for Few-Shot Anomaly Detection

TL;DR

This paper introduces D$^2$4FAD, a dual distillation framework for few-shot anomaly detection in medical imaging, achieving state-of-the-art results with limited normal reference images across diverse organs and modalities.

Contribution

The paper proposes a novel dual distillation approach with a learn-to-weight mechanism for improved few-shot anomaly detection in medical images.

Findings

Outperforms existing methods on a large multi-organ benchmark

Achieves significant improvements in anomaly detection accuracy

Demonstrates robustness across different organs and imaging modalities

Abstract

Anomaly detection is a critical task in computer vision with profound implications for medical imaging, where identifying pathologies early can directly impact patient outcomes. While recent unsupervised anomaly detection approaches show promise, they require substantial normal training data and struggle to generalize across anatomical contexts. We introduce D$^2$4FAD, a novel dual distillation framework for few-shot anomaly detection that identifies anomalies in previously unseen tasks using only a small number of normal reference images. Our approach leverages a pre-trained encoder as a teacher network to extract multi-scale features from both support and query images, while a student decoder learns to distill knowledge from the teacher on query images and self-distill on support images. We further propose a learn-to-weight mechanism that dynamically assesses the reference value of each support image conditioned on the query, optimizing anomaly detection performance. To evaluate our method, we curate a comprehensive benchmark dataset comprising 13,084 images across four organs, four imaging modalities, and five disease categories. Extensive experiments demonstrate that D$^2$4FAD significantly outperforms existing approaches, establishing a new state-of-the-art in few-shot medical anomaly detection. Code is available at https://github.com/ttttqz/D24FAD.