ClickSeg: 3D Instance Segmentation with Click-Level Weak Annotations

TL;DR

ClickSeg introduces a weakly supervised 3D instance segmentation method that requires only one annotated point per object, significantly reducing labeling effort while maintaining high accuracy.

Contribution

The paper proposes a novel click-level weak supervision framework for 3D instance segmentation, utilizing a new training scheme and fixed-seed k-means clustering for pseudo label generation.

Findings

Surpasses previous weakly supervised methods by +9.4% mAP on ScanNetV2.

Achieves about 90% of fully-supervised accuracy with only 0.02% supervision.

Sets new state-of-the-art in weakly supervised 3D semantic segmentation.

Abstract

3D instance segmentation methods often require fully-annotated dense labels for training, which are costly to obtain. In this paper, we present ClickSeg, a novel click-level weakly supervised 3D instance segmentation method that requires one point per instance annotation merely. Such a problem is very challenging due to the extremely limited labels, which has rarely been solved before. We first develop a baseline weakly-supervised training method, which generates pseudo labels for unlabeled data by the model itself. To utilize the property of click-level annotation setting, we further propose a new training framework. Instead of directly using the model inference way, i.e., mean-shift clustering, to generate the pseudo labels, we propose to use k-means with fixed initial seeds: the annotated points. New similarity metrics are further designed for clustering. Experiments on ScanNetV2 and S3DIS datasets show that the proposed ClickSeg surpasses the previous best weakly supervised instance segmentation result by a large margin (e.g., +9.4% mAP on ScanNetV2). Using 0.02% supervision signals merely, ClickSeg achieves $\sim$90% of the accuracy of the fully-supervised counterpart. Meanwhile, it also achieves state-of-the-art semantic segmentation results among weakly supervised methods that use the same annotation settings.