Adapt Before Comparison: A New Perspective on Cross-Domain Few-Shot Segmentation

TL;DR

This paper proposes a test-time task adaptation method for cross-domain few-shot segmentation that outperforms existing methods by eliminating the training stage and using consistency guidance, achieving state-of-the-art results.

Contribution

It introduces a novel test-time adaptation approach that leverages small networks and consistency regularization, removing the need for training on source data.

Findings

Achieves state-of-the-art performance in CD-FSS

Eliminates the training stage for cross-domain segmentation

Uses consistency across augmented views for guidance

Abstract

Few-shot segmentation performance declines substantially when facing images from a domain different than the training domain, effectively limiting real-world use cases. To alleviate this, recently cross-domain few-shot segmentation (CD-FSS) has emerged. Works that address this task mainly attempted to learn segmentation on a source domain in a manner that generalizes across domains. Surprisingly, we can outperform these approaches while eliminating the training stage and removing their main segmentation network. We show test-time task-adaption is the key for successful CD-FSS instead. Task-adaption is achieved by appending small networks to the feature pyramid of a conventionally classification-pretrained backbone. To avoid overfitting to the few labeled samples in supervised fine-tuning, consistency across augmented views of input images serves as guidance while learning the parameters of the attached layers. Despite our self-restriction not to use any images other than the few labeled samples at test time, we achieve new state-of-the-art performance in CD-FSS, evidencing the need to rethink approaches for the task.