DistillFSS: Synthesizing Few-Shot Knowledge into a Lightweight Segmentation Model

TL;DR

DistillFSS introduces a lightweight, support-image-free segmentation model via teacher-student distillation, enabling efficient adaptation to unseen domains and classes with minimal computational cost.

Contribution

It proposes a novel distillation-based framework that internalizes few-shot knowledge into model parameters, eliminating the need for support images during inference.

Findings

Outperforms state-of-the-art methods in multi-class, multi-shot scenarios.

Reduces computational overhead significantly.

Works effectively across diverse domains like medical imaging and remote sensing.

Abstract

Cross-Domain Few-Shot Semantic Segmentation (CD-FSS) seeks to segment unknown classes in unseen domains using only a few annotated examples. This setting is inherently challenging: source and target domains exhibit substantial distribution shifts, label spaces are disjoint, and support images are scarce--making standard episodic methods unreliable and computationally demanding at test time. To address these constraints, we propose DistillFSS, a framework that embeds support-set knowledge directly into a model's parameters through a teacher--student distillation process. By internalizing few-shot reasoning into a dedicated layer within the student network, DistillFSS eliminates the need for support images at test time, enabling fast, lightweight inference, while allowing efficient extension to novel classes in unseen domains through rapid teacher-driven specialization. Combined with fine-tuning, the approach scales efficiently to large support sets and significantly reduces computational overhead. To evaluate the framework under realistic conditions, we introduce a new CD-FSS benchmark spanning medical imaging, industrial inspection, and remote sensing, with disjoint label spaces and variable support sizes. Experiments show that DistillFSS matches or surpasses state-of-the-art baselines, particularly in multi-class and multi-shot scenarios, while offering substantial efficiency gains. The code is available at https://github.com/pasqualedem/DistillFSS.