Unify the Views: View-Consistent Prototype Learning for Few-Shot Segmentation

TL;DR

This paper introduces VINE, a unified framework for few-shot segmentation that models structural consistency and view-invariance to improve prototype quality under viewpoint variations.

Contribution

VINE jointly models structural and view-invariant features using a spatial-view graph and foreground discrimination, enhancing prototype robustness in few-shot segmentation.

Findings

VINE outperforms existing methods on multiple benchmarks.

It demonstrates robustness under large viewpoint shifts.

The framework effectively integrates structural and view-invariant features.

Abstract

Few-shot segmentation (FSS) has gained significant attention for its ability to generalize to novel classes with limited supervision, yet remains challenged by structural misalignment and cross-view inconsistency under large appearance or viewpoint variations. This paper tackles these challenges by introducing VINE (View-Informed NEtwork), a unified framework that jointly models structural consistency and foreground discrimination to refine class-specific prototypes. Specifically, VINE introduces a spatial-view graph on backbone features, where the spatial graph captures local geometric topology and the view graph connects features from different perspectives to propagate view-invariant structural semantics. To further alleviate foreground ambiguity, we derive a discriminative prior from the support-query feature discrepancy to capture category-specific contrast, which reweights SAM features by emphasizing salient regions and recalibrates backbone activations for improved structural focus. The foreground-enhanced SAM features and structurally enriched ResNet features are progressively integrated through masked cross-attention, yielding class-consistent prototypes used as adaptive prompts for the SAM decoder to generate accurate masks. Extensive experiments on multiple FSS benchmarks validate the effectiveness and robustness of VINE, particularly under challenging scenarios with viewpoint shifts and complex structures. The code is available at https://github.com/HongliLiu1/VINE-main.