Towards Part-Based Understanding of RGB-D Scans

TL;DR

This paper introduces a new task of part-based scene understanding from RGB-D scans, focusing on predicting object decompositions into geometric parts using a part graph representation, advancing detailed 3D scene analysis.

Contribution

It proposes a novel framework that leverages part graphs and priors for accurate geometric part prediction, improving over existing methods in 3D scene understanding.

Findings

Part graph guidance improves part completion accuracy.

Part priors enhance geometric decomposition robustness.

Method outperforms alternative approaches in experiments.

Abstract

Recent advances in 3D semantic scene understanding have shown impressive progress in 3D instance segmentation, enabling object-level reasoning about 3D scenes; however, a finer-grained understanding is required to enable interactions with objects and their functional understanding. Thus, we propose the task of part-based scene understanding of real-world 3D environments: from an RGB-D scan of a scene, we detect objects, and for each object predict its decomposition into geometric part masks, which composed together form the complete geometry of the observed object. We leverage an intermediary part graph representation to enable robust completion as well as building of part priors, which we use to construct the final part mask predictions. Our experiments demonstrate that guiding part understanding through part graph to part prior-based predictions significantly outperforms alternative approaches to the task of semantic part completion.