Category-Level 3D Non-Rigid Registration from Single-View RGB Images
Diego Rodriguez, Florian Huber, Sven Behnke

TL;DR
This paper introduces a CNN-based method for 3D non-rigid registration from single-view RGB images, enabling shape reconstruction without depth data and outperforming traditional methods like CPD.
Contribution
The novel approach infers deformation fields directly from RGB images using learned shape spaces, eliminating the need for depth sensors in 3D registration.
Findings
Outperforms CPD in registration accuracy for tested categories
Works effectively on transparent and shiny objects without depth data
Reconstructs 3D models from single RGB images
Abstract
In this paper, we propose a novel approach to solve the 3D non-rigid registration problem from RGB images using Convolutional Neural Networks (CNNs). Our objective is to find a deformation field (typically used for transferring knowledge between instances, e.g., grasping skills) that warps a given 3D canonical model into a novel instance observed by a single-view RGB image. This is done by training a CNN that infers a deformation field for the visible parts of the canonical model and by employing a learned shape (latent) space for inferring the deformations of the occluded parts. As result of the registration, the observed model is reconstructed. Because our method does not need depth information, it can register objects that are typically hard to perceive with RGB-D sensors, e.g. with transparent or shiny surfaces. Even without depth data, our approach outperforms the Coherent Point…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
