PUPS: Point Cloud Unified Panoptic Segmentation

TL;DR

PUPS introduces an end-to-end point cloud panoptic segmentation framework that directly predicts semantic and instance groupings using point classifiers, bipartite matching, and transformer refinement, achieving state-of-the-art results.

Contribution

The paper presents a unified, end-to-end approach for point cloud panoptic segmentation that eliminates hand-crafted heuristics and improves grouping accuracy.

Findings

Achieves 1st place on SemanticKITTI leaderboard.

Sets new state-of-the-art on nuScenes dataset.

Reduces computational cost compared to previous methods.

Abstract

Point cloud panoptic segmentation is a challenging task that seeks a holistic solution for both semantic and instance segmentation to predict groupings of coherent points. Previous approaches treat semantic and instance segmentation as surrogate tasks, and they either use clustering methods or bounding boxes to gather instance groupings with costly computation and hand-crafted designs in the instance segmentation task. In this paper, we propose a simple but effective point cloud unified panoptic segmentation (PUPS) framework, which use a set of point-level classifiers to directly predict semantic and instance groupings in an end-to-end manner. To realize PUPS, we introduce bipartite matching to our training pipeline so that our classifiers are able to exclusively predict groupings of instances, getting rid of hand-crafted designs, e.g. anchors and Non-Maximum Suppression (NMS). In order to achieve better grouping results, we utilize a transformer decoder to iteratively refine the point classifiers and develop a context-aware CutMix augmentation to overcome the class imbalance problem. As a result, PUPS achieves 1st place on the leader board of SemanticKITTI panoptic segmentation task and state-of-the-art results on nuScenes.