Model Selection of Anomaly Detectors in the Absence of Labeled Validation Data

TL;DR

This paper introduces SWSA, a framework for selecting image anomaly detectors without labeled validation data by generating synthetic anomalies from normal images, enabling effective model and prompt selection.

Contribution

The paper presents a novel synthetic anomaly generation method for model selection in zero-shot anomaly detection without labeled validation data.

Findings

SWSA effectively selects models matching ground-truth validation.

Synthetic anomalies improve model selection accuracy.

Outperforms baseline selection strategies.

Abstract

Anomaly detection is the task of identifying abnormal samples in large unlabeled datasets. While the advent of foundation models has produced powerful zero-shot anomaly detection methods, their deployment in practice is often hindered by the absence of labeled validation data -- without it, their detection performance cannot be evaluated reliably. In this work, we propose SWSA (Selection With Synthetic Anomalies): a general-purpose framework to select image-based anomaly detectors without labeled validation data. Instead of collecting labeled validation data, we generate synthetic anomalies without any training or fine-tuning, using only a small support set of normal images. Our synthetic anomalies are used to create detection tasks that compose a validation framework for model selection. In an empirical study, we evaluate SWSA with three types of synthetic anomalies and on two selection tasks: model selection of image-based anomaly detectors and prompt selection for CLIP-based anomaly detection. SWSA often selects models and prompts that match selections made with a ground-truth validation set, outperforming baseline selection strategies.