VLN-PETL: Parameter-Efficient Transfer Learning for Vision-and-Language Navigation

TL;DR

This paper introduces VLN-PETL, a parameter-efficient transfer learning approach tailored for vision-and-language navigation tasks, achieving comparable or superior results to full fine-tuning while significantly reducing training costs.

Contribution

The paper presents the first VLN-specific PETL method with novel modules, improving efficiency and performance in VLN tasks compared to existing PETL techniques.

Findings

VLN-PETL achieves comparable or better performance than full fine-tuning.

VLN-PETL outperforms existing PETL methods on four VLN benchmarks.

The proposed modules enhance cross-modal and historical interaction in VLN models.

Abstract

The performance of the Vision-and-Language Navigation~(VLN) tasks has witnessed rapid progress recently thanks to the use of large pre-trained vision-and-language models. However, full fine-tuning the pre-trained model for every downstream VLN task is becoming costly due to the considerable model size. Recent research hotspot of Parameter-Efficient Transfer Learning (PETL) shows great potential in efficiently tuning large pre-trained models for the common CV and NLP tasks, which exploits the most of the representation knowledge implied in the pre-trained model while only tunes a minimal set of parameters. However, simply utilizing existing PETL methods for the more challenging VLN tasks may bring non-trivial degeneration to the performance. Therefore, we present the first study to explore PETL methods for VLN tasks and propose a VLN-specific PETL method named VLN-PETL. Specifically, we design two PETL modules: Historical Interaction Booster (HIB) and Cross-modal Interaction Booster (CIB). Then we combine these two modules with several existing PETL methods as the integrated VLN-PETL. Extensive experimental results on four mainstream VLN tasks (R2R, REVERIE, NDH, RxR) demonstrate the effectiveness of our proposed VLN-PETL, where VLN-PETL achieves comparable or even better performance to full fine-tuning and outperforms other PETL methods with promising margins.