Towards Zero-shot Point Cloud Anomaly Detection: A Multi-View Projection Framework

TL;DR

This paper introduces a multi-view projection framework that leverages pre-trained vision-language models for zero-shot anomaly detection in point clouds, transforming 3D data into images for effective anomaly identification without extensive training.

Contribution

The paper proposes a novel multi-view projection framework that adapts pre-trained vision-language models for zero-shot point cloud anomaly detection, including learnable prompting techniques for improved performance.

Findings

Outperforms existing zero-shot methods on MVTec 3D-AD and Real3D-AD datasets.

Effective in real-world automotive plastic part inspection scenarios.

Learnable prompts significantly enhance detection accuracy.

Abstract

Detecting anomalies within point clouds is crucial for various industrial applications, but traditional unsupervised methods face challenges due to data acquisition costs, early-stage production constraints, and limited generalization across product categories. To overcome these challenges, we introduce the Multi-View Projection (MVP) framework, leveraging pre-trained Vision-Language Models (VLMs) to detect anomalies. Specifically, MVP projects point cloud data into multi-view depth images, thereby translating point cloud anomaly detection into image anomaly detection. Following zero-shot image anomaly detection methods, pre-trained VLMs are utilized to detect anomalies on these depth images. Given that pre-trained VLMs are not inherently tailored for zero-shot point cloud anomaly detection and may lack specificity, we propose the integration of learnable visual and adaptive text prompting techniques to fine-tune these VLMs, thereby enhancing their detection performance. Extensive experiments on the MVTec 3D-AD and Real3D-AD demonstrate our proposed MVP framework's superior zero-shot anomaly detection performance and the prompting techniques' effectiveness. Real-world evaluations on automotive plastic part inspection further showcase that the proposed method can also be generalized to practical unseen scenarios. The code is available at https://github.com/hustCYQ/MVP-PCLIP.