MVP: Meta Visual Prompt Tuning for Few-Shot Remote Sensing Image Scene Classification

TL;DR

This paper introduces MVP, a parameter-efficient meta visual prompt tuning method for few-shot remote sensing image scene classification, combining prompt tuning with meta-learning and a novel data augmentation to improve performance and reduce overfitting.

Contribution

The paper proposes MVP, integrating VPT into meta-learning for remote sensing, and introduces a patch embedding recombination augmentation, addressing overfitting and efficiency issues.

Findings

MVP outperforms existing methods on FS-RSSC benchmark.

Effective in various few-shot and cross-domain scenarios.

Reduces overfitting and storage compared to full fine-tuning.

Abstract

Vision Transformer (ViT) models have recently emerged as powerful and versatile models for various visual tasks. Recently, a work called PMF has achieved promising results in few-shot image classification by utilizing pre-trained vision transformer models. However, PMF employs full fine-tuning for learning the downstream tasks, leading to significant overfitting and storage issues, especially in the remote sensing domain. In order to tackle these issues, we turn to the recently proposed parameter-efficient tuning methods, such as VPT, which updates only the newly added prompt parameters while keeping the pre-trained backbone frozen. Inspired by VPT, we propose the Meta Visual Prompt Tuning (MVP) method. Specifically, we integrate the VPT method into the meta-learning framework and tailor it to the remote sensing domain, resulting in an efficient framework for Few-Shot Remote Sensing Scene Classification (FS-RSSC). Furthermore, we introduce a novel data augmentation strategy based on patch embedding recombination to enhance the representation and diversity of scenes for classification purposes. Experiment results on the FS-RSSC benchmark demonstrate the superior performance of the proposed MVP over existing methods in various settings, such as various-way-various-shot, various-way-one-shot, and cross-domain adaptation.