Semantic Segmentation-Assisted Instance Feature Fusion for Multi-Level 3D Part Instance Segmentation

TL;DR

This paper introduces a novel 3D part instance segmentation method that leverages semantic segmentation for feature fusion and improves clustering, achieving significant performance gains on the PartNet benchmark.

Contribution

It proposes a new semantic segmentation-assisted feature fusion scheme and a semantic region center prediction task to enhance 3D part instance segmentation accuracy.

Findings

Outperforms existing methods on PartNet benchmark

Improves indoor scene instance segmentation performance

Enhances clustering of instance points

Abstract

Recognizing 3D part instances from a 3D point cloud is crucial for 3D structure and scene understanding. Several learning-based approaches use semantic segmentation and instance center prediction as training tasks and fail to further exploit the inherent relationship between shape semantics and part instances. In this paper, we present a new method for 3D part instance segmentation. Our method exploits semantic segmentation to fuse nonlocal instance features, such as center prediction, and further enhances the fusion scheme in a multi- and cross-level way. We also propose a semantic region center prediction task to train and leverage the prediction results to improve the clustering of instance points. Our method outperforms existing methods with a large-margin improvement in the PartNet benchmark. We also demonstrate that our feature fusion scheme can be applied to other existing methods to improve their performance in indoor scene instance segmentation tasks.