Relevant Intrinsic Feature Enhancement Network for Few-Shot Semantic Segmentation

TL;DR

This paper introduces RiFeNet, a novel network for few-shot semantic segmentation that enhances intrinsic feature extraction and inter-class distinction, achieving superior results without extra unlabeled data during testing.

Contribution

The paper proposes RiFeNet with an unlabeled branch for intrinsic feature learning and a multi-level prototype module to improve class distinction in few-shot segmentation.

Findings

Outperforms state-of-the-art on PASCAL-5i and COCO benchmarks

Effectively utilizes unlabeled data during training without extra computation at test time

Enhances intra-class consistency and inter-class variability understanding

Abstract

For few-shot semantic segmentation, the primary task is to extract class-specific intrinsic information from limited labeled data. However, the semantic ambiguity and inter-class similarity of previous methods limit the accuracy of pixel-level foreground-background classification. To alleviate these issues, we propose the Relevant Intrinsic Feature Enhancement Network (RiFeNet). To improve the semantic consistency of foreground instances, we propose an unlabeled branch as an efficient data utilization method, which teaches the model how to extract intrinsic features robust to intra-class differences. Notably, during testing, the proposed unlabeled branch is excluded without extra unlabeled data and computation. Furthermore, we extend the inter-class variability between foreground and background by proposing a novel multi-level prototype generation and interaction module. The different-grained complementarity between global and local prototypes allows for better distinction between similar categories. The qualitative and quantitative performance of RiFeNet surpasses the state-of-the-art methods on PASCAL-5i and COCO benchmarks.