Auto3R: Automated 3D Reconstruction and Scanning via Data-driven Uncertainty Quantification

TL;DR

Auto3R is a data-driven model that automates 3D scanning and reconstruction by predicting uncertainty over viewpoints, enabling robots to efficiently generate accurate and photorealistic 3D models without prior ground truth data.

Contribution

Auto3R introduces a novel uncertainty quantification approach for fully automated 3D scanning and reconstruction, handling complex materials and improving over existing methods.

Findings

Outperforms state-of-the-art methods significantly

Successfully digitizes real-world objects with a robot arm

Produces photorealistic 3D assets

Abstract

Traditional high-quality 3D scanning and reconstruction typically relies on human labor to plan the scanning procedure. With the rapid development of embodied systems such as drones and robots, there is a growing demand of performing accurate 3D scanning and reconstruction in an fully automated manner. We introduce Auto3R, a data-driven uncertainty quantification model that is designed to automate the 3D scanning and reconstruction of scenes and objects, including objects with non-lambertian and specular materials. Specifically, in a process of iterative 3D reconstruction and scanning, Auto3R can make efficient and accurate prediction of uncertainty distribution over potential scanning viewpoints, without knowing the ground truth geometry and appearance. Through extensive experiments, Auto3R achieves superior performance that outperforms the state-of-the-art methods by a large margin. We also deploy Auto3R on a robot arm equipped with a camera and demonstrate that Auto3R can be used to effectively digitize real-world 3D objects and delivers ready-to-use and photorealistic digital assets. Our homepage: https://tomatoma00.github.io/auto3r.github.io .