Triple Point Masking

TL;DR

This paper introduces a triple point masking scheme (TPM) for 3D point cloud autoencoders, enabling multi-mask learning to improve fine-grained recovery and downstream task performance, especially under limited data conditions.

Contribution

The paper proposes TPM, a novel multi-mask learning framework with an SVM-guided weight selection module, enhancing 3D autoencoder pre-training and fine-tuning capabilities.

Findings

Enhanced performance on downstream tasks with TPM

Improved fine-grained 3D object recovery

Versatile framework applicable to various baselines

Abstract

Existing 3D mask learning methods encounter performance bottlenecks under limited data, and our objective is to overcome this limitation. In this paper, we introduce a triple point masking scheme, named TPM, which serves as a scalable framework for pre-training of masked autoencoders to achieve multi-mask learning for 3D point clouds. Specifically, we augment the baselines with two additional mask choices (i.e., medium mask and low mask) as our core insight is that the recovery process of an object can manifest in diverse ways. Previous high-masking schemes focus on capturing the global representation but lack the fine-grained recovery capability, so that the generated pre-trained weights tend to play a limited role in the fine-tuning process. With the support of the proposed TPM, available methods can exhibit more flexible and accurate completion capabilities, enabling the potential autoencoder in the pre-training stage to consider multiple representations of a single 3D object. In addition, an SVM-guided weight selection module is proposed to fill the encoder parameters for downstream networks with the optimal weight during the fine-tuning stage, maximizing linear accuracy and facilitating the acquisition of intricate representations for new objects. Extensive experiments show that the four baselines equipped with the proposed TPM achieve comprehensive performance improvements on various downstream tasks. Our code and models are available at https://github.com/liujia99/TPM.