Shape Anchor Guided Holistic Indoor Scene Understanding

TL;DR

This paper introduces AncLearn, a shape anchor guided learning strategy that improves indoor scene understanding by reducing noise in detection and reconstruction, leading to state-of-the-art results on ScanNetv2.

Contribution

We propose a novel shape anchor guided learning strategy (AncLearn) that enhances proposal generation and object sampling for better scene understanding.

Findings

Achieves state-of-the-art 3D detection accuracy on ScanNetv2.

Improves mesh reconstruction quality without segmentation.

Effectively reduces noise in proposal features and point sampling.

Abstract

This paper proposes a shape anchor guided learning strategy (AncLearn) for robust holistic indoor scene understanding. We observe that the search space constructed by current methods for proposal feature grouping and instance point sampling often introduces massive noise to instance detection and mesh reconstruction. Accordingly, we develop AncLearn to generate anchors that dynamically fit instance surfaces to (i) unmix noise and target-related features for offering reliable proposals at the detection stage, and (ii) reduce outliers in object point sampling for directly providing well-structured geometry priors without segmentation during reconstruction. We embed AncLearn into a reconstruction-from-detection learning system (AncRec) to generate high-quality semantic scene models in a purely instance-oriented manner. Experiments conducted on the challenging ScanNetv2 dataset demonstrate that our shape anchor-based method consistently achieves state-of-the-art performance in terms of 3D object detection, layout estimation, and shape reconstruction. The code will be available at https://github.com/Geo-Tell/AncRec.