Is Training Necessary for Anomaly Detection?

TL;DR

This paper introduces Retrieval-based Anomaly Detection (RAD), a training-free, memory-based approach that outperforms traditional reconstruction-based methods in unsupervised anomaly detection tasks.

Contribution

The paper proposes RAD, a novel training-free anomaly detection method using memory retrieval, challenging the necessity of training in MUAD approaches.

Findings

RAD achieves state-of-the-art results on multiple benchmarks.

RAD performs well with minimal data, reaching high accuracy with just one image.

Retrieval scores upper-bound reconstruction residuals, providing theoretical insights.

Abstract

Current state-of-the-art multi-class unsupervised anomaly detection (MUAD) methods rely on training encoder-decoder models to reconstruct anomaly-free features. We first show these approaches have an inherent fidelity-stability dilemma in how they detect anomalies via reconstruction residuals. We then abandon the reconstruction paradigm entirely and propose Retrieval-based Anomaly Detection (RAD). RAD is a training-free approach that stores anomaly-free features in a memory and detects anomalies through multi-level retrieval, matching test patches against the memory. Experiments demonstrate that RAD achieves state-of-the-art performance across four established benchmarks (MVTec-AD, VisA, Real-IAD, 3D-ADAM) under both standard and few-shot settings. On MVTec-AD, RAD reaches 96.7\% Pixel AUROC with just a single anomaly-free image compared to 98.5\% of RAD's full-data performance. We further prove that retrieval-based scores theoretically upper-bound reconstruction-residual scores. Collectively, these findings overturn the assumption that MUAD requires task-specific training, showing that state-of-the-art anomaly detection is feasible with memory-based retrieval. Our code is available at https://github.com/longkukuhi/RAD.