Exploiting Point-Language Models with Dual-Prompts for 3D Anomaly Detection

TL;DR

This paper introduces PLANE, a dual-prompt point-language model for 3D anomaly detection that generalizes across multiple categories with a single model, using dynamic prompts and pseudo 3D data augmentation.

Contribution

The paper presents a novel dual-prompt learning framework with dynamic prompt creation and pseudo 3D anomaly generation for improved multi-category 3D anomaly detection.

Findings

Achieves +8.7%/+17% detection and localization gains on Anomaly-ShapeNet.

Obtains +4.3%/+4.1% gains on Real3D-AD dataset.

Operates effectively under a multi-class-one-model paradigm.

Abstract

Anomaly detection (AD) in 3D point clouds is crucial in a wide range of industrial applications, especially in various forms of precision manufacturing. Considering the industrial demand for reliable 3D AD, several methods have been developed. However, most of these approaches typically require training separate models for each category, which is memory-intensive and lacks flexibility. In this paper, we propose a novel Point-Language model with dual-prompts for 3D ANomaly dEtection (PLANE). The approach leverages multi-modal prompts to extend the strong generalization capabilities of pre-trained Point-Language Models (PLMs) to the domain of 3D point cloud AD, achieving impressive detection performance across multiple categories using a single model. Specifically, we propose a dual-prompt learning method, incorporating both text and point cloud prompts. The method utilizes a dynamic prompt creator module (DPCM) to produce sample-specific dynamic prompts, which are then integrated with class-specific static prompts for each modality, effectively driving the PLMs. Additionally, based on the characteristics of point cloud data, we propose a pseudo 3D anomaly generation method (Ano3D) to improve the model's detection capabilities in an unsupervised setting. Experimental results demonstrate that the proposed method, which is under the multi-class-one-model paradigm, achieves a +8.7%/+17% gain on anomaly detection and localization performance as compared to the state-of-the-art one-class-one-model methods for the Anomaly-ShapeNet dataset, and obtains +4.3%/+4.1% gain for the Real3D-AD dataset. Code will be available upon publication.