Back to Point: Exploring Point-Language Models for Zero-Shot 3D Anomaly Detection

TL;DR

This paper introduces BTP, a novel framework that uses pre-trained Point-Language Models for zero-shot 3D anomaly detection, effectively aligning 3D point cloud features with textual descriptions to improve detection sensitivity and robustness.

Contribution

The paper proposes BTP, a new approach that aligns 3D point cloud features with language embeddings, incorporating geometric descriptors and joint learning for enhanced zero-shot anomaly detection.

Findings

BTP outperforms existing methods on Real3D-AD and Anomaly-ShapeNet datasets.

Effective alignment of multi-granularity features improves localized anomaly detection.

Incorporating geometric descriptors enhances sensitivity to structural anomalies.

Abstract

Zero-shot (ZS) 3D anomaly detection is crucial for reliable industrial inspection, as it enables detecting and localizing defects without requiring any target-category training data. Existing approaches render 3D point clouds into 2D images and leverage pre-trained Vision-Language Models (VLMs) for anomaly detection. However, such strategies inevitably discard geometric details and exhibit limited sensitivity to local anomalies. In this paper, we revisit intrinsic 3D representations and explore the potential of pre-trained Point-Language Models (PLMs) for ZS 3D anomaly detection. We propose BTP (Back To Point), a novel framework that effectively aligns 3D point cloud and textual embeddings. Specifically, BTP aligns multi-granularity patch features with textual representations for localized anomaly detection, while incorporating geometric descriptors to enhance sensitivity to structural anomalies. Furthermore, we introduce a joint representation learning strategy that leverages auxiliary point cloud data to improve robustness and enrich anomaly semantics. Extensive experiments on Real3D-AD and Anomaly-ShapeNet demonstrate that BTP achieves superior performance in ZS 3D anomaly detection. Code will be available at \href{https://github.com/wistful-8029/BTP-3DAD}{https://github.com/wistful-8029/BTP-3DAD}.