Looking 3D: Anomaly Detection with 2D-3D Alignment

TL;DR

This paper presents a new transformer-based method for anomaly detection by aligning 2D images with 3D reference shapes, supported by a large dataset, to improve detection accuracy in manufacturing contexts.

Contribution

Introduces a novel conditional anomaly detection framework using 2D-3D alignment and a new dataset, along with a transformer-based approach for improved detection performance.

Findings

Effective 2D-3D alignment improves anomaly detection accuracy.

The proposed method outperforms existing techniques on the new dataset.

The dataset serves as a benchmark for future research in this area.

Abstract

Automatic anomaly detection based on visual cues holds practical significance in various domains, such as manufacturing and product quality assessment. This paper introduces a new conditional anomaly detection problem, which involves identifying anomalies in a query image by comparing it to a reference shape. To address this challenge, we have created a large dataset, BrokenChairs-180K, consisting of around 180K images, with diverse anomalies, geometries, and textures paired with 8,143 reference 3D shapes. To tackle this task, we have proposed a novel transformer-based approach that explicitly learns the correspondence between the query image and reference 3D shape via feature alignment and leverages a customized attention mechanism for anomaly detection. Our approach has been rigorously evaluated through comprehensive experiments, serving as a benchmark for future research in this domain.