In-Hand 3D Object Scanning from an RGB Sequence

TL;DR

This paper introduces a novel in-hand 3D object scanning method using a monocular camera that jointly optimizes shape and pose without prior pose information, enabling accurate reconstruction of textured and texture-less objects.

Contribution

It presents an incremental optimization approach that jointly estimates shape and pose from RGB sequences without known camera poses, improving robustness and accuracy.

Findings

Successfully reconstructs textured and texture-less objects.

Outperforms classical appearance-based methods.

Achieves performance close to pose-aware recent methods.

Abstract

We propose a method for in-hand 3D scanning of an unknown object with a monocular camera. Our method relies on a neural implicit surface representation that captures both the geometry and the appearance of the object, however, by contrast with most NeRF-based methods, we do not assume that the camera-object relative poses are known. Instead, we simultaneously optimize both the object shape and the pose trajectory. As direct optimization over all shape and pose parameters is prone to fail without coarse-level initialization, we propose an incremental approach that starts by splitting the sequence into carefully selected overlapping segments within which the optimization is likely to succeed. We reconstruct the object shape and track its poses independently within each segment, then merge all the segments before performing a global optimization. We show that our method is able to reconstruct the shape and color of both textured and challenging texture-less objects, outperforms classical methods that rely only on appearance features, and that its performance is close to recent methods that assume known camera poses.