Relation3D: Enhancing Relation Modeling for Point Cloud Instance Segmentation

TL;DR

Relation3D introduces advanced relation modeling techniques, including superpoint aggregation and a relation-aware self-attention mechanism, to significantly improve 3D instance segmentation accuracy on multiple datasets.

Contribution

The paper proposes novel modules for superpoint feature representation and a relation-aware self-attention mechanism, addressing limitations of existing transformer-based methods in modeling internal scene relationships.

Findings

Achieves superior performance on ScanNetV2, ScanNet++, ScanNet200, and S3DIS datasets.

Effectively models internal relationships among scene features and between query features.

Outperforms existing methods in 3D instance segmentation accuracy.

Abstract

3D instance segmentation aims to predict a set of object instances in a scene, representing them as binary foreground masks with corresponding semantic labels. Currently, transformer-based methods are gaining increasing attention due to their elegant pipelines and superior predictions. However, these methods primarily focus on modeling the external relationships between scene features and query features through mask attention. They lack effective modeling of the internal relationships among scene features as well as between query features. In light of these disadvantages, we propose \textbf{Relation3D: Enhancing Relation Modeling for Point Cloud Instance Segmentation}. Specifically, we introduce an adaptive superpoint aggregation module and a contrastive learning-guided superpoint refinement module to better represent superpoint features (scene features) and leverage contrastive learning to guide the updates of these features. Furthermore, our relation-aware self-attention mechanism enhances the capabilities of modeling relationships between queries by incorporating positional and geometric relationships into the self-attention mechanism. Extensive experiments on the ScanNetV2, ScanNet++, ScanNet200 and S3DIS datasets demonstrate the superior performance of Relation3D.