GP-S3Net: Graph-based Panoptic Sparse Semantic Segmentation Network

TL;DR

GP-S3Net introduces a novel, efficient graph-based LiDAR panoptic segmentation method that eliminates the need for object proposals, achieving state-of-the-art results on multiple datasets.

Contribution

It proposes a proposal-free, graph convolutional network approach for LiDAR panoptic segmentation, enhancing efficiency and accuracy over traditional two-stage methods.

Findings

Outperforms current state-of-the-art methods on nuScenes and SemanticPOSS datasets.

Achieves first place on the SemanticKITTI leaderboard.

Demonstrates significant accuracy improvements with a proposal-free framework.

Abstract

Panoptic segmentation as an integrated task of both static environmental understanding and dynamic object identification, has recently begun to receive broad research interest. In this paper, we propose a new computationally efficient LiDAR based panoptic segmentation framework, called GP-S3Net. GP-S3Net is a proposal-free approach in which no object proposals are needed to identify the objects in contrast to conventional two-stage panoptic systems, where a detection network is incorporated for capturing instance information. Our new design consists of a novel instance-level network to process the semantic results by constructing a graph convolutional network to identify objects (foreground), which later on are fused with the background classes. Through the fine-grained clusters of the foreground objects from the semantic segmentation backbone, over-segmentation priors are generated and subsequently processed by 3D sparse convolution to embed each cluster. Each cluster is treated as a node in the graph and its corresponding embedding is used as its node feature. Then a GCNN predicts whether edges exist between each cluster pair. We utilize the instance label to generate ground truth edge labels for each constructed graph in order to supervise the learning. Extensive experiments demonstrate that GP-S3Net outperforms the current state-of-the-art approaches, by a significant margin across available datasets such as, nuScenes and SemanticPOSS, ranking first on the competitive public SemanticKITTI leaderboard upon publication.