Multi-similarity based Hyperrelation Network for few-shot segmentation

TL;DR

This paper introduces MSHNet, a novel network leveraging multi-similarity hyperrelations and a generative prototype similarity to improve few-shot semantic segmentation, achieving state-of-the-art results on Pascal-5i and COCO-20i datasets.

Contribution

The paper proposes a new multi-similarity hyperrelation framework with GPS and SMB modules, enhancing semantic relation modeling and reducing overfitting in few-shot segmentation.

Findings

MSHNet achieves state-of-the-art performance on Pascal-5i and COCO-20i datasets.

The combination of GPS and cosine similarity improves semantic relation modeling.

Efficient merging of multi-layer and multi-shot features enhances segmentation accuracy.

Abstract

Few-shot semantic segmentation aims at recognizing the object regions of unseen categories with only a few annotated examples as supervision. The key to few-shot segmentation is to establish a robust semantic relationship between the support and query images and to prevent overfitting. In this paper, we propose an effective Multi-similarity Hyperrelation Network (MSHNet) to tackle the few-shot semantic segmentation problem. In MSHNet, we propose a new Generative Prototype Similarity (GPS), which together with cosine similarity can establish a strong semantic relation between the support and query images. The locally generated prototype similarity based on global feature is logically complementary to the global cosine similarity based on local feature, and the relationship between the query image and the supported image can be expressed more comprehensively by using the two similarities simultaneously. In addition, we propose a Symmetric Merging Block (SMB) in MSHNet to efficiently merge multi-layer, multi-shot and multi-similarity hyperrelational features. MSHNet is built on the basis of similarity rather than specific category features, which can achieve more general unity and effectively reduce overfitting. On two benchmark semantic segmentation datasets Pascal-5i and COCO-20i, MSHNet achieves new state-of-the-art performances on 1-shot and 5-shot semantic segmentation tasks.