Uncertainty-aware 3D Object-Level Mapping with Deep Shape Priors

TL;DR

This paper introduces an uncertainty-aware framework for 3D object mapping that reconstructs high-quality shapes and poses of unknown objects from RGB-D images, leveraging deep shape priors and probabilistic optimization.

Contribution

It presents a novel probabilistic, uncertainty-aware approach for 3D object reconstruction that explicitly models shape and pose uncertainties during optimization.

Findings

Outperforms state-of-the-art methods on real-world datasets.

Effectively models shape and pose uncertainties reflecting true errors.

Enhances downstream robotics tasks like active vision.

Abstract

3D object-level mapping is a fundamental problem in robotics, which is especially challenging when object CAD models are unavailable during inference. In this work, we propose a framework that can reconstruct high-quality object-level maps for unknown objects. Our approach takes multiple RGB-D images as input and outputs dense 3D shapes and 9-DoF poses (including 3 scale parameters) for detected objects. The core idea of our approach is to leverage a learnt generative model for shape categories as a prior and to formulate a probabilistic, uncertainty-aware optimization framework for 3D reconstruction. We derive a probabilistic formulation that propagates shape and pose uncertainty through two novel loss functions. Unlike current state-of-the-art approaches, we explicitly model the uncertainty of the object shapes and poses during our optimization, resulting in a high-quality object-level mapping system. Moreover, the resulting shape and pose uncertainties, which we demonstrate can accurately reflect the true errors of our object maps, can also be useful for downstream robotics tasks such as active vision. We perform extensive evaluations on indoor and outdoor real-world datasets, achieving achieves substantial improvements over state-of-the-art methods. Our code will be available at https://github.com/TRAILab/UncertainShapePose.