PointCMP: Contrastive Mask Prediction for Self-supervised Learning on Point Cloud Videos

TL;DR

PointCMP introduces a contrastive mask prediction framework for self-supervised learning on point cloud videos, effectively capturing local and global spatio-temporal features, leading to state-of-the-art results and robust transfer learning performance.

Contribution

The paper proposes a novel two-branch contrastive learning framework with a mutual similarity augmentation module for point cloud videos, enhancing representation quality without labels.

Findings

Achieves state-of-the-art performance on benchmark datasets.

Outperforms existing fully supervised methods.

Demonstrates strong transfer learning capabilities.

Abstract

Self-supervised learning can extract representations of good quality from solely unlabeled data, which is appealing for point cloud videos due to their high labelling cost. In this paper, we propose a contrastive mask prediction (PointCMP) framework for self-supervised learning on point cloud videos. Specifically, our PointCMP employs a two-branch structure to achieve simultaneous learning of both local and global spatio-temporal information. On top of this two-branch structure, a mutual similarity based augmentation module is developed to synthesize hard samples at the feature level. By masking dominant tokens and erasing principal channels, we generate hard samples to facilitate learning representations with better discrimination and generalization performance. Extensive experiments show that our PointCMP achieves the state-of-the-art performance on benchmark datasets and outperforms existing full-supervised counterparts. Transfer learning results demonstrate the superiority of the learned representations across different datasets and tasks.