Step-wise Distribution Alignment Guided Style Prompt Tuning for Source-free Cross-domain Few-shot Learning

TL;DR

This paper introduces StepSPT, a novel style prompt tuning method for source-free cross-domain few-shot learning that effectively reduces domain gaps using a step-wise distribution alignment strategy, without source data or extensive training.

Contribution

The paper proposes a new style prompt tuning approach with a step-wise distribution alignment strategy for source-free cross-domain few-shot learning, addressing data access and computational challenges.

Findings

Outperforms existing prompt tuning methods and SOTA on five datasets.

Effectively narrows domain gaps through prediction distribution optimization.

Ablation studies confirm the effectiveness of the proposed method.

Abstract

Existing cross-domain few-shot learning (CDFSL) methods, which develop source-domain training strategies to enhance model transferability, face challenges with large-scale pre-trained models (LMs) due to inaccessible source data and training strategies. Moreover, fine-tuning LMs for CDFSL demands substantial computational resources, limiting practicality. This paper addresses the source-free CDFSL (SF-CDFSL) problem, tackling few-shot learning (FSL) in the target domain using only pre-trained models and a few target samples without source data or strategies. To overcome the challenge of inaccessible source data, this paper introduces Step-wise Distribution Alignment Guided Style Prompt Tuning (StepSPT), which implicitly narrows domain gaps through prediction distribution optimization. StepSPT proposes a style prompt to align target samples with the desired distribution and adopts a dual-phase optimization process. In the external process, a step-wise distribution alignment strategy factorizes prediction distribution optimization into a multi-step alignment problem to tune the style prompt. In the internal process, the classifier is updated using standard cross-entropy loss. Evaluations on five datasets demonstrate that StepSPT outperforms existing prompt tuning-based methods and SOTAs. Ablation studies further verify its effectiveness. Code will be made publicly available at https://github.com/xuhuali-mxj/StepSPT.