Joint prototype and coefficient prediction for 3D instance segmentation

TL;DR

This paper presents a novel 3D instance segmentation method that learns prototypes and coefficients simultaneously, producing interpretable predictions with improved accuracy and speed, suitable for real-time applications.

Contribution

The proposed method introduces joint prototype and coefficient prediction with an overcomplete sampling strategy and NMS, achieving faster and more reliable 3D instance segmentation.

Findings

Outperforms existing methods in mRec and mPrec on S3DIS datasets

Operates 32.9% faster than the state-of-the-art

Reduces inference time variance by over 20-fold

Abstract

3D instance segmentation is crucial for applications demanding comprehensive 3D scene understanding. In this paper, we introduce a novel method that simultaneously learns coefficients and prototypes. Employing an overcomplete sampling strategy, our method produces an overcomplete set of instance predictions, from which the optimal ones are selected through a Non-Maximum Suppression (NMS) algorithm during inference. The obtained prototypes are visualizable and interpretable. Our method demonstrates superior performance on S3DIS-blocks, consistently outperforming existing methods in mRec and mPrec. Moreover, it operates 32.9% faster than the state-of-the-art. Notably, with only 0.8% of the total inference time, our method exhibits an over 20-fold reduction in the variance of inference time compared to existing methods. These attributes render our method well-suited for practical applications requiring both rapid inference and high reliability.