Hierarchical Aggregation for 3D Instance Segmentation

TL;DR

This paper introduces HAIS, a hierarchical clustering framework for 3D instance segmentation on point clouds, achieving state-of-the-art accuracy efficiently without non-maximum suppression.

Contribution

The paper presents a novel hierarchical aggregation approach that improves clustering-based 3D instance segmentation, with high efficiency and accuracy.

Findings

Achieves 69.9% AP50 on ScanNet v2, surpassing previous SOTA.

Fast processing time of 410ms per frame.

Demonstrates strong generalization on S3DIS dataset.

Abstract

Instance segmentation on point clouds is a fundamental task in 3D scene perception. In this work, we propose a concise clustering-based framework named HAIS, which makes full use of spatial relation of points and point sets. Considering clustering-based methods may result in over-segmentation or under-segmentation, we introduce the hierarchical aggregation to progressively generate instance proposals, i.e., point aggregation for preliminarily clustering points to sets and set aggregation for generating complete instances from sets. Once the complete 3D instances are obtained, a sub-network of intra-instance prediction is adopted for noisy points filtering and mask quality scoring. HAIS is fast (only 410ms per frame) and does not require non-maximum suppression. It ranks 1st on the ScanNet v2 benchmark, achieving the highest 69.9% AP50 and surpassing previous state-of-the-art (SOTA) methods by a large margin. Besides, the SOTA results on the S3DIS dataset validate the good generalization ability. Code will be available at https://github.com/hustvl/HAIS.