Masked Clustering Prediction for Unsupervised Point Cloud Pre-training

TL;DR

MaskClu is an unsupervised pre-training method for vision transformers on 3D point clouds that combines masked point modeling, clustering, and contrastive learning to capture dense semantic features, improving performance across various 3D tasks.

Contribution

It introduces MaskClu, a novel approach integrating clustering-based reconstruction and contrastive learning for better semantic understanding in point cloud ViTs.

Findings

Outperforms existing methods on part and semantic segmentation

Achieves state-of-the-art results in 3D object detection

Enhances semantic feature richness in point cloud representations

Abstract

Vision transformers (ViTs) have recently been widely applied to 3D point cloud understanding, with masked autoencoding as the predominant pre-training paradigm. However, the challenge of learning dense and informative semantic features from point clouds via standard ViTs remains underexplored. We propose MaskClu, a novel unsupervised pre-training method for ViTs on 3D point clouds that integrates masked point modeling with clustering-based learning. MaskClu is designed to reconstruct both cluster assignments and cluster centers from masked point clouds, thus encouraging the model to capture dense semantic information. Additionally, we introduce a global contrastive learning mechanism that enhances instance-level feature learning by contrasting different masked views of the same point cloud. By jointly optimizing these complementary objectives, i.e., dense semantic reconstruction, and instance-level contrastive learning. MaskClu enables ViTs to learn richer and more semantically meaningful representations from 3D point clouds. We validate the effectiveness of our method via multiple 3D tasks, including part segmentation, semantic segmentation, object detection, and classification, where MaskClu sets new competitive results. The code and models will be released at:https://github.com/Amazingren/maskclu.