You Only Click Once: Single Point Weakly Supervised 3D Instance Segmentation for Autonomous Driving

TL;DR

This paper introduces YoCo, a weakly supervised 3D instance segmentation framework for autonomous driving that uses minimal annotations and combines vision models with geometric and temporal constraints to achieve near fully supervised performance.

Contribution

YoCo is the first framework to generate high-quality pseudo labels from sparse click annotations using vision models and geometric constraints, reducing annotation effort significantly.

Findings

Achieves state-of-the-art results among weakly supervised methods.

Surpasses fully supervised Cylinder3D with minimal fine-tuning.

Uses only 0.8% of fully labeled data for comparable performance.

Abstract

Outdoor LiDAR point cloud 3D instance segmentation is a crucial task in autonomous driving. However, it requires laborious human efforts to annotate the point cloud for training a segmentation model. To address this challenge, we propose a YoCo framework, which generates 3D pseudo labels using minimal coarse click annotations in the bird's eye view plane. It is a significant challenge to produce high-quality pseudo labels from sparse annotations. Our YoCo framework first leverages vision foundation models combined with geometric constraints from point clouds to enhance pseudo label generation. Second, a temporal and spatial-based label updating module is designed to generate reliable updated labels. It leverages predictions from adjacent frames and utilizes the inherent density variation of point clouds (dense near, sparse far). Finally, to further improve label quality, an IoU-guided enhancement module is proposed, replacing pseudo labels with high-confidence and high-IoU predictions. Experiments on the Waymo dataset demonstrate YoCo's effectiveness and generality, achieving state-of-the-art performance among weakly supervised methods and surpassing fully supervised Cylinder3D. Additionally, the YoCo is suitable for various networks, achieving performance comparable to fully supervised methods with minimal fine-tuning using only 0.8% of the fully labeled data, significantly reducing annotation costs.