MRAD: Zero-Shot Anomaly Detection with Memory-Driven Retrieval

TL;DR

MRAD introduces a memory retrieval-based framework for zero-shot anomaly detection that outperforms existing methods by leveraging explicit data distribution without complex modeling.

Contribution

The paper proposes MRAD, a novel train-free memory retrieval framework for zero-shot anomaly detection, replacing parametric fitting with explicit memory banks for improved cross-domain stability.

Findings

Superior performance on 16 datasets in anomaly detection tasks.

Effective zero-shot detection without training, using memory retrieval.

Enhanced generalization via lightweight variants MRAD-FT and MRAD-CLIP.

Abstract

Zero-shot anomaly detection (ZSAD) often leverages pretrained vision or vision-language models, but many existing methods use prompt learning or complex modeling to fit the data distribution, resulting in high training or inference cost and limited cross-domain stability. To address these limitations, we propose Memory-Retrieval Anomaly Detection method (MRAD), a unified framework that replaces parametric fitting with a direct memory retrieval. The train-free base model, MRAD-TF, freezes the CLIP image encoder and constructs a two-level memory bank (image-level and pixel-level) from auxiliary data, where feature-label pairs are explicitly stored as keys and values. During inference, anomaly scores are obtained directly by similarity retrieval over the memory bank. Based on the MRAD-TF, we further propose two lightweight variants as enhancements: (i) MRAD-FT fine-tunes the retrieval metric with two linear layers to enhance the discriminability between normal and anomaly; (ii) MRAD-CLIP injects the normal and anomalous region priors from the MRAD-FT as dynamic biases into CLIP's learnable text prompts, strengthening generalization to unseen categories. Across 16 industrial and medical datasets, the MRAD framework consistently demonstrates superior performance in anomaly classification and segmentation, under both train-free and training-based settings. Our work shows that fully leveraging the empirical distribution of raw data, rather than relying only on model fitting, can achieve stronger anomaly detection performance. The code will be publicly released at https://github.com/CROVO1026/MRAD.