ZeroReg: Zero-Shot Point Cloud Registration with Foundation Models
Weijie Wang, Wenqi Ren, Guofeng Mei, Bin Ren, Xiaoshui Huang, Fabio, Poiesi, Nicu Sebe, Bruno Lepri

TL;DR
ZeroReg introduces a zero-shot 3D point cloud registration method leveraging foundation models for semantic feature extraction and scene graph matching, enabling effective registration without training on labeled datasets.
Contribution
It is the first to utilize 2D foundation models for zero-shot 3D registration, combining semantic features and scene graph matching for improved accuracy.
Findings
Competitive performance on 3DMatch, 3DLoMatch, and ScanNet benchmarks.
Effective object-to-point matching using foundation model features.
Robust registration achieved without labeled training data.
Abstract
State-of-the-art 3D point cloud registration methods rely on labeled 3D datasets for training, which limits their practical applications in real-world scenarios and often hinders generalization to unseen scenes. Leveraging the zero-shot capabilities of foundation models offers a promising solution to these challenges. In this paper, we introduce ZeroReg, a zero-shot registration approach that utilizes 2D foundation models to predict 3D correspondences. Specifically, ZeroReg adopts an object-to-point matching strategy, starting with object localization and semantic feature extraction from multi-view images using foundation models. In the object matching stage, semantic features help identify correspondences between objects across views. However, relying solely on semantic features can lead to ambiguity, especially in scenes with multiple instances of the same category. To address this,…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
Topics3D Shape Modeling and Analysis · 3D Surveying and Cultural Heritage · Robotics and Sensor-Based Localization
