Learning to Complete Object Shapes for Object-level Mapping in Dynamic Scenes

TL;DR

This paper introduces an object-level mapping system that segments, tracks, and reconstructs objects in dynamic scenes, improving accuracy by predicting complete geometries using shape priors and depth inputs.

Contribution

It presents a novel system that jointly optimizes object pose and shape, integrating shape completion with tracking and mapping in dynamic environments.

Findings

Outperforms traditional volumetric mapping methods in accuracy

Effective in both synthetic and real-world sequences

Enhances object reconstruction quality through shape prior conditioning

Abstract

In this paper, we propose a novel object-level mapping system that can simultaneously segment, track, and reconstruct objects in dynamic scenes. It can further predict and complete their full geometries by conditioning on reconstructions from depth inputs and a category-level shape prior with the aim that completed object geometry leads to better object reconstruction and tracking accuracy. For each incoming RGB-D frame, we perform instance segmentation to detect objects and build data associations between the detection and the existing object maps. A new object map will be created for each unmatched detection. For each matched object, we jointly optimise its pose and latent geometry representations using geometric residual and differential rendering residual towards its shape prior and completed geometry. Our approach shows better tracking and reconstruction performance compared to methods using traditional volumetric mapping or learned shape prior approaches. We evaluate its effectiveness by quantitatively and qualitatively testing it in both synthetic and real-world sequences.