BridgeNet: A Unified Multimodal Framework for Bridging 2D and 3D Industrial Anomaly Detection

TL;DR

BridgeNet introduces a unified multimodal framework that effectively bridges 2D and 3D anomaly detection by disentangling depth and appearance, generating richer anomalies, and sharing parameters across modalities, leading to superior performance.

Contribution

The paper presents a novel framework that disentangles depth and appearance, employs multimodal anomaly generation, and shares parameters to unify 2D and 3D anomaly detection.

Findings

Outperforms state-of-the-art on MVTec-3D AD and Eyecandies datasets.

Effectively generates richer anomalies in RGB and depth modalities.

Bridges 2D and 3D anomaly detection without complex fusion.

Abstract

Industrial anomaly detection for 2D objects has gained significant attention and achieved progress in anomaly detection (AD) methods. However, identifying 3D depth anomalies using only 2D information is insufficient. Despite explicitly fusing depth information into RGB images or using point cloud backbone networks to extract depth features, both approaches struggle to adequately represent 3D information in multimodal scenarios due to the disparities among different modal information. Additionally, due to the scarcity of abnormal samples in industrial data, especially in multimodal scenarios, it is necessary to perform anomaly generation to simulate real-world abnormal samples. Therefore, we propose a novel unified multimodal anomaly detection framework to address these issues. Our contributions consist of 3 key aspects. (1) We extract visible depth information from 3D point cloud data simply and use 2D RGB images to represent appearance, which disentangles depth and appearance to support unified anomaly generation. (2) Benefiting from the flexible input representation, the proposed Multi-Scale Gaussian Anomaly Generator and Unified Texture Anomaly Generator can generate richer anomalies in RGB and depth. (3) All modules share parameters for both RGB and depth data, effectively bridging 2D and 3D anomaly detection. Subsequent modules can directly leverage features from both modalities without complex fusion. Experiments show our method outperforms state-of-the-art (SOTA) on MVTec-3D AD and Eyecandies datasets. Code available at: https://github.com/Xantastic/BridgeNet