Associatively Segmenting Instances and Semantics in Point Clouds

TL;DR

This paper introduces a flexible framework for simultaneous instance and semantic segmentation in 3D point clouds, leveraging mutual benefits between the two tasks to improve accuracy and outperform existing methods.

Contribution

The paper proposes a novel approach that enables instance and semantic segmentation to enhance each other, achieving superior results in 3D point cloud analysis.

Findings

Significant improvement over state-of-the-art in 3D instance segmentation

Notable enhancement in 3D semantic segmentation accuracy

Framework effectively leverages mutual task benefits

Abstract

A 3D point cloud describes the real scene precisely and intuitively.To date how to segment diversified elements in such an informative 3D scene is rarely discussed. In this paper, we first introduce a simple and flexible framework to segment instances and semantics in point clouds simultaneously. Then, we propose two approaches which make the two tasks take advantage of each other, leading to a win-win situation. Specifically, we make instance segmentation benefit from semantic segmentation through learning semantic-aware point-level instance embedding. Meanwhile, semantic features of the points belonging to the same instance are fused together to make more accurate per-point semantic predictions. Our method largely outperforms the state-of-the-art method in 3D instance segmentation along with a significant improvement in 3D semantic segmentation. Code has been made available at: https://github.com/WXinlong/ASIS.