FreePoint: Unsupervised Point Cloud Instance Segmentation

TL;DR

FreePoint introduces an unsupervised, class-agnostic point cloud instance segmentation framework that leverages self-supervised features and a multicut algorithm, significantly reducing the need for manual annotations and outperforming existing methods.

Contribution

The paper presents a novel unsupervised framework for point cloud instance segmentation using a multicut algorithm and a weakly-supervised training strategy, achieving state-of-the-art results.

Findings

Outperforms previous methods by over 18.2% in AP.

Achieves 5.5% higher AP than UnScene3D.

Surpasses existing self-supervised pre-training methods by 6.0% in AP with limited annotations.

Abstract

Instance segmentation of point clouds is a crucial task in 3D field with numerous applications that involve localizing and segmenting objects in a scene. However, achieving satisfactory results requires a large number of manual annotations, which is a time-consuming and expensive process. To alleviate dependency on annotations, we propose a novel framework, FreePoint, for underexplored unsupervised class-agnostic instance segmentation on point clouds. In detail, we represent the point features by combining coordinates, colors, and self-supervised deep features. Based on the point features, we perform a bottom-up multicut algorithm to segment point clouds into coarse instance masks as pseudo labels, which are used to train a point cloud instance segmentation model. We propose an id-as-feature strategy at this stage to alleviate the randomness of the multicut algorithm and improve the pseudo labels' quality. During training, we propose a weakly-supervised two-step training strategy and corresponding losses to overcome the inaccuracy of coarse masks. FreePoint has achieved breakthroughs in unsupervised class-agnostic instance segmentation on point clouds and outperformed previous traditional methods by over 18.2% and a competitive concurrent work UnScene3D by 5.5% in AP. Additionally, when used as a pretext task and fine-tuned on S3DIS, FreePoint performs significantly better than existing self-supervised pre-training methods with limited annotations and surpasses CSC by 6.0% in AP with 10% annotation masks.