SiM3D: Single-instance Multiview Multimodal and Multisetup 3D Anomaly Detection Benchmark

TL;DR

SiM3D introduces a comprehensive benchmark for 3D anomaly detection that integrates multiview and multimodal data, focusing on single-instance scenarios and generalizing from synthetic to real data, with a new dataset and baseline evaluations.

Contribution

It presents the first benchmark for multiview multimodal 3D anomaly detection in single-instance scenarios, including a new dataset and adapted baseline methods.

Findings

Baseline methods show room for improvement on the new benchmark.

Multiview and multimodal integration improves anomaly detection performance.

The dataset enables evaluation of generalization from synthetic to real data.

Abstract

We propose SiM3D, the first benchmark considering the integration of multiview and multimodal information for comprehensive 3D anomaly detection and segmentation (ADS), where the task is to produce a voxel-based Anomaly Volume. Moreover, SiM3D focuses on a scenario of high interest in manufacturing: single-instance anomaly detection, where only one object, either real or synthetic, is available for training. In this respect, SiM3D stands out as the first ADS benchmark that addresses the challenge of generalising from synthetic training data to real test data. SiM3D includes a novel multimodal multiview dataset acquired using top-tier industrial sensors and robots. The dataset features multiview high-resolution images (12 Mpx) and point clouds (7M points) for 333 instances of eight types of objects, alongside a CAD model for each type. We also provide manually annotated 3D segmentation GTs for anomalous test samples. To establish reference baselines for the proposed multiview 3D ADS task, we adapt prominent singleview methods and assess their performance using novel metrics that operate on Anomaly Volumes.