RAD: A Comprehensive Dataset for Benchmarking the Robustness of Image Anomaly Detection

TL;DR

This paper introduces the RAD dataset to evaluate the robustness of image anomaly detection methods under real-world noisy conditions, highlighting the impact of various noise factors and the effectiveness of certain approaches.

Contribution

The study provides a comprehensive dataset with diverse noise conditions for benchmarking anomaly detection robustness and analyzes the performance of 11 state-of-the-art methods under these conditions.

Findings

Memory bank-based methods are more robust to noise.

Synthetic anomalies improve detection robustness.

Foundational models can enhance anomaly detection robustness.

Abstract

Robustness against noisy imaging is crucial for practical image anomaly detection systems. This study introduces a Robust Anomaly Detection (RAD) dataset with free views, uneven illuminations, and blurry collections to systematically evaluate the robustness of current anomaly detection methods. Specifically, RAD aims to identify foreign objects on working platforms as anomalies. The collection process incorporates various sources of imaging noise, such as viewpoint changes, uneven illuminations, and blurry collections, to replicate real-world inspection scenarios. Subsequently, we assess and analyze 11 state-of-the-art unsupervised and zero-shot methods on RAD. Our findings indicate that: 1) Variations in viewpoint, illumination, and blurring affect anomaly detection methods to varying degrees; 2) Methods relying on memory banks and assisted by synthetic anomalies demonstrate stronger robustness; 3) Effectively leveraging the general knowledge of foundational models is a promising avenue for enhancing the robustness of anomaly detection methods. The dataset is available at https://github.com/hustCYQ/RAD-dataset.