DBGroup: Dual-Branch Point Grouping for Weakly Supervised 3D Semantic Instance Segmentation

TL;DR

DBGroup introduces a dual-branch point grouping framework for weakly supervised 3D instance segmentation, utilizing scene-level annotations and multi-view cues to generate and refine pseudo labels, achieving competitive results with reduced annotation effort.

Contribution

The paper proposes a novel two-stage weakly supervised 3D instance segmentation method using scene-level annotations and a dual-branch point grouping module for pseudo label generation and refinement.

Findings

Achieves competitive performance with sparse-point supervision.

Surpasses state-of-the-art scene-level supervised 3D segmentation.

Effective pseudo label refinement strategies improve segmentation quality.

Abstract

Weakly supervised 3D instance segmentation is essential for 3D scene understanding, especially as the growing scale of data and high annotation costs associated with fully supervised approaches. Existing methods primarily rely on two forms of weak supervision: one-thing-one-click annotations and bounding box annotations, both of which aim to reduce labeling efforts. However, these approaches still encounter limitations, including labor-intensive annotation processes, high complexity, and reliance on expert annotators. To address these challenges, we propose \textbf{DBGroup}, a two-stage weakly supervised 3D instance segmentation framework that leverages scene-level annotations as a more efficient and scalable alternative. In the first stage, we introduce a Dual-Branch Point Grouping module to generate pseudo labels guided by semantic and mask cues extracted from multi-view images. To further improve label quality, we develop two refinement strategies: Granularity-Aware Instance Merging and Semantic Selection and Propagation. The second stage involves multi-round self-training on an end-to-end instance segmentation network using the refined pseudo-labels. Additionally, we introduce an Instance Mask Filter strategy to address inconsistencies within the pseudo labels. Extensive experiments demonstrate that DBGroup achieves competitive performance compared to sparse-point-level supervised 3D instance segmentation methods, while surpassing state-of-the-art scene-level supervised 3D semantic segmentation approaches. Code is available at https://github.com/liuxuexun/DBGroup.