Self-Support Few-Shot Semantic Segmentation

TL;DR

This paper introduces a novel self-support matching strategy for few-shot semantic segmentation that leverages query predictions to improve prototype quality and better capture intra-class variations, achieving state-of-the-art results.

Contribution

It proposes a self-support matching strategy using query prototypes, along with an adaptive background prototype generation and self-support loss, to enhance few-shot segmentation performance.

Findings

Substantially improves prototype quality.

Achieves state-of-the-art results on multiple datasets.

Benefits more from stronger backbones and additional supports.

Abstract

Existing few-shot segmentation methods have achieved great progress based on the support-query matching framework. But they still heavily suffer from the limited coverage of intra-class variations from the few-shot supports provided. Motivated by the simple Gestalt principle that pixels belonging to the same object are more similar than those to different objects of same class, we propose a novel self-support matching strategy to alleviate this problem, which uses query prototypes to match query features, where the query prototypes are collected from high-confidence query predictions. This strategy can effectively capture the consistent underlying characteristics of the query objects, and thus fittingly match query features. We also propose an adaptive self-support background prototype generation module and self-support loss to further facilitate the self-support matching procedure. Our self-support network substantially improves the prototype quality, benefits more improvement from stronger backbones and more supports, and achieves SOTA on multiple datasets. Codes are at \url{https://github.com/fanq15/SSP}.