Real-IAD: A Real-World Multi-View Dataset for Benchmarking Versatile Industrial Anomaly Detection

TL;DR

This paper introduces Real-IAD, a large-scale, multi-view dataset for industrial anomaly detection, addressing current dataset limitations and proposing a new fully unsupervised setting to better reflect real-world industrial scenarios.

Contribution

The paper presents the Real-IAD dataset with 150K images, multi-view data collection, and sample-level metrics, along with a new Fully Unsupervised Industrial Anomaly Detection setting.

Findings

Existing methods saturate on current datasets, limiting differentiation.

Real-IAD is significantly larger and more challenging than previous datasets.

Benchmark results highlight the difficulty of the new dataset and setting.

Abstract

Industrial anomaly detection (IAD) has garnered significant attention and experienced rapid development. However, the recent development of IAD approach has encountered certain difficulties due to dataset limitations. On the one hand, most of the state-of-the-art methods have achieved saturation (over 99% in AUROC) on mainstream datasets such as MVTec, and the differences of methods cannot be well distinguished, leading to a significant gap between public datasets and actual application scenarios. On the other hand, the research on various new practical anomaly detection settings is limited by the scale of the dataset, posing a risk of overfitting in evaluation results. Therefore, we propose a large-scale, Real-world, and multi-view Industrial Anomaly Detection dataset, named Real-IAD, which contains 150K high-resolution images of 30 different objects, an order of magnitude larger than existing datasets. It has a larger range of defect area and ratio proportions, making it more challenging than previous datasets. To make the dataset closer to real application scenarios, we adopted a multi-view shooting method and proposed sample-level evaluation metrics. In addition, beyond the general unsupervised anomaly detection setting, we propose a new setting for Fully Unsupervised Industrial Anomaly Detection (FUIAD) based on the observation that the yield rate in industrial production is usually greater than 60%, which has more practical application value. Finally, we report the results of popular IAD methods on the Real-IAD dataset, providing a highly challenging benchmark to promote the development of the IAD field.