SVasP: Self-Versatility Adversarial Style Perturbation for Cross-Domain Few-Shot Learning

TL;DR

This paper introduces SVasP, a novel style perturbation method for cross-domain few-shot learning that improves model robustness and transferability by stabilizing gradients and diversifying style representations.

Contribution

SVasP proposes a self-versatility style perturbation technique that enhances gradient stability and escapes poor minima, improving cross-domain few-shot learning performance.

Findings

Outperforms existing state-of-the-art methods on multiple benchmarks.

Enhances gradient stability and model generalization.

Boosts transferability to unseen target domains.

Abstract

Cross-Domain Few-Shot Learning (CD-FSL) aims to transfer knowledge from seen source domains to unseen target domains, which is crucial for evaluating the generalization and robustness of models. Recent studies focus on utilizing visual styles to bridge the domain gap between different domains. However, the serious dilemma of gradient instability and local optimization problem occurs in those style-based CD-FSL methods. This paper addresses these issues and proposes a novel crop-global style perturbation method, called \underline{\textbf{S}}elf-\underline{\textbf{V}}ersatility \underline{\textbf{A}}dversarial \underline{\textbf{S}}tyle \underline{\textbf{P}}erturbation (\textbf{SVasP}), which enhances the gradient stability and escapes from poor sharp minima jointly. Specifically, SVasP simulates more diverse potential target domain adversarial styles via diversifying input patterns and aggregating localized crop style gradients, to serve as global style perturbation stabilizers within one image, a concept we refer to as self-versatility. Then a novel objective function is proposed to maximize visual discrepancy while maintaining semantic consistency between global, crop, and adversarial features. Having the stabilized global style perturbation in the training phase, one can obtain a flattened minima in the loss landscape, boosting the transferability of the model to the target domains. Extensive experiments on multiple benchmark datasets demonstrate that our method significantly outperforms existing state-of-the-art methods. Our codes are available at https://github.com/liwenqianSEU/SVasP.