MaskGroup: Hierarchical Point Grouping and Masking for 3D Instance Segmentation

TL;DR

This paper introduces MaskGroup, a hierarchical point grouping and masking framework for 3D instance segmentation that improves accuracy by multi-scale grouping and mask refinement.

Contribution

It proposes a novel hierarchical grouping algorithm and MaskScoreNet for refined 3D instance segmentation, outperforming previous methods.

Findings

Achieves 66.4% mAP on ScanNetV2, surpassing state-of-the-art by 1.9%.

Effectively handles objects of different scales through multi-scale grouping.

Demonstrates superior segmentation accuracy on benchmark datasets.

Abstract

This paper studies the 3D instance segmentation problem, which has a variety of real-world applications such as robotics and augmented reality. Since the surroundings of 3D objects are of high complexity, the separating of different objects is very difficult. To address this challenging problem, we propose a novel framework to group and refine the 3D instances. In practice, we first learn an offset vector for each point and shift it to its predicted instance center. To better group these points, we propose a Hierarchical Point Grouping algorithm to merge the centrally aggregated points progressively. All points are grouped into small clusters, which further gradually undergo another clustering procedure to merge into larger groups. These multi-scale groups are exploited for instance prediction, which is beneficial for predicting instances with different scales. In addition, a novel MaskScoreNet is developed to produce binary point masks of these groups for further refining the segmentation results. Extensive experiments conducted on the ScanNetV2 and S3DIS benchmarks demonstrate the effectiveness of the proposed method. For instance, our approach achieves a 66.4\% mAP with the 0.5 IoU threshold on the ScanNetV2 test set, which is 1.9\% higher than the state-of-the-art method.