Forecast-PEFT: Parameter-Efficient Fine-Tuning for Pre-trained Motion Forecasting Models

TL;DR

Forecast-PEFT introduces a parameter-efficient fine-tuning method for pre-trained motion forecasting models, focusing on prompts and adapters to improve efficiency and performance without extensive retraining.

Contribution

The paper proposes Forecast-PEFT, a novel fine-tuning strategy that freezes most parameters and fine-tunes only prompts and adapters, significantly reducing training costs while maintaining high accuracy.

Findings

Outperforms full fine-tuning with only 17% of parameters trained.

Achieves up to 9.6% improvement with Forecast-FT.

Enhances dataset adaptation and model robustness.

Abstract

Recent progress in motion forecasting has been substantially driven by self-supervised pre-training. However, adapting pre-trained models for specific downstream tasks, especially motion prediction, through extensive fine-tuning is often inefficient. This inefficiency arises because motion prediction closely aligns with the masked pre-training tasks, and traditional full fine-tuning methods fail to fully leverage this alignment. To address this, we introduce Forecast-PEFT, a fine-tuning strategy that freezes the majority of the model's parameters, focusing adjustments on newly introduced prompts and adapters. This approach not only preserves the pre-learned representations but also significantly reduces the number of parameters that need retraining, thereby enhancing efficiency. This tailored strategy, supplemented by our method's capability to efficiently adapt to different datasets, enhances model efficiency and ensures robust performance across datasets without the need for extensive retraining. Our experiments show that Forecast-PEFT outperforms traditional full fine-tuning methods in motion prediction tasks, achieving higher accuracy with only 17% of the trainable parameters typically required. Moreover, our comprehensive adaptation, Forecast-FT, further improves prediction performance, evidencing up to a 9.6% enhancement over conventional baseline methods. Code will be available at https://github.com/csjfwang/Forecast-PEFT.