Self-supervised adversarial masking for 3D point cloud representation learning

TL;DR

This paper introduces PointCAM, an adversarial self-supervised learning method for 3D point clouds that learns to select optimal masks, leading to improved representation learning and performance on downstream tasks.

Contribution

It proposes a novel adversarial masking approach with an online tokenizer and self-distillation, surpassing random masking methods in 3D point cloud representation learning.

Findings

Achieves state-of-the-art performance on downstream tasks

Learned masking function outperforms random masking

Demonstrates effectiveness across various 3D datasets

Abstract

Self-supervised methods have been proven effective for learning deep representations of 3D point cloud data. Although recent methods in this domain often rely on random masking of inputs, the results of this approach can be improved. We introduce PointCAM, a novel adversarial method for learning a masking function for point clouds. Our model utilizes a self-distillation framework with an online tokenizer for 3D point clouds. Compared to previous techniques that optimize patch-level and object-level objectives, we postulate applying an auxiliary network that learns how to select masks instead of choosing them randomly. Our results show that the learned masking function achieves state-of-the-art or competitive performance on various downstream tasks. The source code is available at https://github.com/szacho/pointcam.