DQFormer: Towards Unified LiDAR Panoptic Segmentation with Decoupled Queries

TL;DR

DQFormer introduces a unified LiDAR panoptic segmentation framework that decouples queries for things and stuff, improving segmentation accuracy by addressing their intrinsic differences and reducing ambiguity.

Contribution

The paper proposes a novel decoupled query-based framework, DQFormer, for LiDAR panoptic segmentation, effectively handling things and stuff separately within a unified model.

Findings

Outperforms existing methods on nuScenes and SemanticKITTI datasets.

Effectively mitigates mutual competition and ambiguity in segmentation.

Demonstrates superior accuracy and robustness in LiDAR perception tasks.

Abstract

LiDAR panoptic segmentation, which jointly performs instance and semantic segmentation for things and stuff classes, plays a fundamental role in LiDAR perception tasks. While most existing methods explicitly separate these two segmentation tasks and utilize different branches (i.e., semantic and instance branches), some recent methods have embraced the query-based paradigm to unify LiDAR panoptic segmentation. However, the distinct spatial distribution and inherent characteristics of objects(things) and their surroundings(stuff) in 3D scenes lead to challenges, including the mutual competition of things/stuff and the ambiguity of classification/segmentation. In this paper, we propose decoupling things/stuff queries according to their intrinsic properties for individual decoding and disentangling classification/segmentation to mitigate ambiguity. To this end, we propose a novel framework dubbed DQFormer to implement semantic and instance segmentation in a unified workflow. Specifically, we design a decoupled query generator to propose informative queries with semantics by localizing things/stuff positions and fusing multi-level BEV embeddings. Moreover, a query-oriented mask decoder is introduced to decode corresponding segmentation masks by performing masked cross-attention between queries and mask embeddings. Finally, the decoded masks are combined with the semantics of the queries to produce panoptic results. Extensive experiments on nuScenes and SemanticKITTI datasets demonstrate the superiority of our DQFormer framework.