RestNet: Boosting Cross-Domain Few-Shot Segmentation with Residual Transformation Network

TL;DR

RestNet introduces a residual transformation network that enhances cross-domain few-shot segmentation by preserving intra-domain information and utilizing semantic and prototype-based modules, achieving state-of-the-art results without extra fine-tuning.

Contribution

The paper proposes RestNet, a novel network with SEAT and IRE modules, for improved cross-domain few-shot segmentation that retains intra-domain features and transfers knowledge effectively.

Findings

Achieves state-of-the-art performance on multiple datasets.

Effectively preserves intra-domain information during transfer.

No additional fine-tuning required for new domains.

Abstract

Cross-domain few-shot segmentation (CD-FSS) aims to achieve semantic segmentation in previously unseen domains with a limited number of annotated samples. Although existing CD-FSS models focus on cross-domain feature transformation, relying exclusively on inter-domain knowledge transfer may lead to the loss of critical intra-domain information. To this end, we propose a novel residual transformation network (RestNet) that facilitates knowledge transfer while retaining the intra-domain support-query feature information. Specifically, we propose a Semantic Enhanced Anchor Transform (SEAT) module that maps features to a stable domain-agnostic space using advanced semantics. Additionally, an Intra-domain Residual Enhancement (IRE) module is designed to maintain the intra-domain representation of the original discriminant space in the new space. We also propose a mask prediction strategy based on prototype fusion to help the model gradually learn how to segment. Our RestNet can transfer cross-domain knowledge from both inter-domain and intra-domain without requiring additional fine-tuning. Extensive experiments on ISIC, Chest X-ray, and FSS-1000 show that our RestNet achieves state-of-the-art performance. Our code will be available soon.