Fence off Anomaly Interference: Cross-Domain Distillation for Fully Unsupervised Anomaly Detection

TL;DR

This paper introduces a novel cross-domain distillation framework for fully unsupervised anomaly detection, effectively handling training data with anomalies by dividing data into domains and aggregating knowledge to improve detection accuracy.

Contribution

It pioneers the use of cross-domain knowledge distillation with domain-specific students to enhance FUAD performance in the presence of anomalous training data.

Findings

Significant performance improvements on noisy MVTec AD and VisA datasets.

Effective handling of training data with anomalies through domain division.

Validation of the proposed method's robustness and effectiveness.

Abstract

Fully Unsupervised Anomaly Detection (FUAD) is a practical extension of Unsupervised Anomaly Detection (UAD), aiming to detect anomalies without any labels even when the training set may contain anomalous samples. To achieve FUAD, we pioneer the introduction of Knowledge Distillation (KD) paradigm based on teacher-student framework into the FUAD setting. However, due to the presence of anomalies in the training data, traditional KD methods risk enabling the student to learn the teacher's representation of anomalies under FUAD setting, thereby resulting in poor anomaly detection performance. To address this issue, we propose a novel Cross-Domain Distillation (CDD) framework based on the widely studied reverse distillation (RD) paradigm. Specifically, we design a Domain-Specific Training, which divides the training set into multiple domains with lower anomaly ratios and train a domain-specific student for each. Cross-Domain Knowledge Aggregation is then performed, where pseudo-normal features generated by domain-specific students collaboratively guide a global student to learn generalized normal representations across all samples. Experimental results on noisy versions of the MVTec AD and VisA datasets demonstrate that our method achieves significant performance improvements over the baseline, validating its effectiveness under FUAD setting.