Dynamic Adapter Meets Prompt Tuning: Parameter-Efficient Transfer Learning for Point Cloud Analysis

TL;DR

This paper introduces DAPT, a parameter-efficient transfer learning method for point cloud analysis that combines Dynamic Adapter and Prompt Tuning to reduce training costs while maintaining high performance.

Contribution

It proposes a novel Dynamic Adapter integrated with Prompt Tuning for efficient model adaptation in point cloud analysis, significantly reducing parameters and training memory.

Findings

DAPT outperforms full fine-tuning on five datasets.

Reduces trainable parameters by 95%.

Cuts training GPU memory by 35%.

Abstract

Point cloud analysis has achieved outstanding performance by transferring point cloud pre-trained models. However, existing methods for model adaptation usually update all model parameters, i.e., full fine-tuning paradigm, which is inefficient as it relies on high computational costs (e.g., training GPU memory) and massive storage space. In this paper, we aim to study parameter-efficient transfer learning for point cloud analysis with an ideal trade-off between task performance and parameter efficiency. To achieve this goal, we freeze the parameters of the default pre-trained models and then propose the Dynamic Adapter, which generates a dynamic scale for each token, considering the token significance to the downstream task. We further seamlessly integrate Dynamic Adapter with Prompt Tuning (DAPT) by constructing Internal Prompts, capturing the instance-specific features for interaction. Extensive experiments conducted on five challenging datasets demonstrate that the proposed DAPT achieves superior performance compared to the full fine-tuning counterparts while significantly reducing the trainable parameters and training GPU memory by 95% and 35%, respectively. Code is available at https://github.com/LMD0311/DAPT.