DAP-MAE: Domain-Adaptive Point Cloud Masked Autoencoder for Effective Cross-Domain Learning

TL;DR

DAP-MAE is a novel pre-training approach for 3D point clouds that adaptively integrates cross-domain knowledge, significantly improving performance on multiple downstream tasks by employing domain-specific adapters and feature guidance.

Contribution

It introduces a domain-adaptive autoencoder with a heterogeneous domain adapter and feature generator for effective cross-domain point cloud learning.

Findings

Achieves 95.18% object classification accuracy on ScanObjectNN.

Attains 88.45% facial expression recognition accuracy on Bosphorus.

Demonstrates strong performance across four diverse point cloud analysis tasks.

Abstract

Compared to 2D data, the scale of point cloud data in different domains available for training, is quite limited. Researchers have been trying to combine these data of different domains for masked autoencoder (MAE) pre-training to leverage such a data scarcity issue. However, the prior knowledge learned from mixed domains may not align well with the downstream 3D point cloud analysis tasks, leading to degraded performance. To address such an issue, we propose the Domain-Adaptive Point Cloud Masked Autoencoder (DAP-MAE), an MAE pre-training method, to adaptively integrate the knowledge of cross-domain datasets for general point cloud analysis. In DAP-MAE, we design a heterogeneous domain adapter that utilizes an adaptation mode during pre-training, enabling the model to comprehensively learn information from point clouds across different domains, while employing a fusion mode in the fine-tuning to enhance point cloud features. Meanwhile, DAP-MAE incorporates a domain feature generator to guide the adaptation of point cloud features to various downstream tasks. With only one pre-training, DAP-MAE achieves excellent performance across four different point cloud analysis tasks, reaching 95.18% in object classification on ScanObjectNN and 88.45% in facial expression recognition on Bosphorus.