An Adapter-free Fine-tuning Approach for Tuning 3D Foundation Models

TL;DR

This paper introduces MCFT, an adapter-free fine-tuning method for 3D foundation models that improves adaptation in low-data scenarios without increasing inference latency, outperforming prior approaches.

Contribution

The paper proposes MCFT, a novel adapter-free fine-tuning technique that maintains model efficiency while enhancing performance on 3D tasks, with semi-supervised and pruning extensions.

Findings

MCFT outperforms prior methods in object recognition and segmentation tasks.

Semi-supervised MCFT achieves up to 6.13% improvement in few-shot learning.

MCFT maintains inference efficiency with no additional parameters.

Abstract

Point cloud foundation models demonstrate strong generalization, yet adapting them to downstream tasks remains challenging in low-data regimes. Full fine-tuning often leads to overfitting and significant drift from pre-trained representations, while existing parameter-efficient fine-tuning (PEFT) methods mitigate this issue by introducing additional trainable components at the cost of increased inference-time latency. We propose Momentum-Consistency Fine-Tuning (MCFT), an adapter-free approach that bridges the gap between full and parameter-efficient fine-tuning. MCFT selectively fine-tunes a portion of the pre-trained encoder while enforcing a momentum-based consistency constraint to preserve task-agnostic representations. Unlike PEFT methods, MCFT introduces no additional representation learning parameters beyond a standard task head, maintaining the original model's parameter count and inference efficiency. We further extend MCFT with two variants: a semi-supervised framework that leverages abundant unlabeled data to enhance few-shot performance, and a pruning-based variant that improves computational efficiency through structured layer removal. Extensive experiments on object recognition and part segmentation benchmarks demonstrate that MCFT consistently outperforms prior methods, achieving a 3.30% gain in 5-shot settings and up to a 6.13% improvement with semi-supervised learning, while remaining well-suited for resource-constrained deployment.