Training-Free Fine-Grained Semantic Segmentations in Low Data Regimes: A FungiTastic Baseline

TL;DR

This paper introduces a training-free, two-stage framework for fine-grained semantic segmentation in low-data regimes, combining macro-taxonomic prompts with prototype matching to achieve scalable and effective results.

Contribution

It presents the first training-free baseline for fine-grained semantic segmentation in low-data settings, decoupling segmentation from classification with a novel feature space transformation.

Findings

Effective in one-shot to few-hundred-shot regimes

Improves prototype classification with feature space transformation

Scalable approach with low segmentation cost

Abstract

Fine-grained semantic segmentation requires both precise localization and discrimination between visually similar classes. In FungiTastic, this problem is further complicated by a long-tailed distribution and strong variation in image acquisition conditions. We propose a training-free two-stage framework that decouples segmentation from classification. SAM3 first produces class-agnostic mushroom masks using macro-taxonomic prompts, and DINOv3 then assigns fine-grained labels through prototype matching in the embedding space. To improve this stage, we apply a simple transformation of the DINOv3 feature space that improves prototype-based classification. Compared with class-specific prompting, our approach is more scalable and keeps the segmentation cost low. We report results from one-shot to few-hundred-shot regimes, providing, to the best of our knowledge, the first baseline for fine-grained semantic segmentation in low-data settings.