Zero-Shot Image Anomaly Detection Using Generative Foundation Models

TL;DR

This paper introduces a novel zero-shot anomaly detection method using diffusion models as perceptual templates, achieving state-of-the-art results without dataset-specific re-training.

Contribution

It presents a new approach leveraging denoising diffusion models and Stein score errors for zero-shot anomaly detection, outperforming existing methods.

Findings

Near-perfect performance on some benchmarks

Effective use of CelebA as a base distribution

Outperforms models trained on ImageNet in certain settings

Abstract

Detecting out-of-distribution (OOD) inputs is pivotal for deploying safe vision systems in open-world environments. We revisit diffusion models, not as generators, but as universal perceptual templates for OOD detection. This research explores the use of score-based generative models as foundational tools for semantic anomaly detection across unseen datasets. Specifically, we leverage the denoising trajectories of Denoising Diffusion Models (DDMs) as a rich source of texture and semantic information. By analyzing Stein score errors, amplified through the Structural Similarity Index Metric (SSIM), we introduce a novel method for identifying anomalous samples without requiring re-training on each target dataset. Our approach improves over state-of-the-art and relies on training a single model on one dataset -- CelebA -- which we find to be an effective base distribution, even outperforming more commonly used datasets like ImageNet in several settings. Experimental results show near-perfect performance on some benchmarks, with notable headroom on others, highlighting both the strength and future potential of generative foundation models in anomaly detection.