Overcoming Support Dilution for Robust Few-shot Semantic Segmentation

TL;DR

This paper addresses the support dilution problem in few-shot semantic segmentation by proposing methods to identify, preserve, and enhance high-contributed supports, leading to improved segmentation performance especially with larger support sets.

Contribution

The authors introduce a contribution index, a Symmetric Correlation module, and a support image pruning technique to effectively focus on high-contributed supports in FSS.

Findings

Significant performance improvements on COCO-20i and PASCAL-5i benchmarks.

Effective support selection enhances segmentation accuracy.

Method demonstrates practical real-world segmentation capabilities.

Abstract

Few-shot Semantic Segmentation (FSS) is a challenging task that utilizes limited support images to segment associated unseen objects in query images. However, recent FSS methods are observed to perform worse, when enlarging the number of shots. As the support set enlarges, existing FSS networks struggle to concentrate on the high-contributed supports and could easily be overwhelmed by the low-contributed supports that could severely impair the mask predictions. In this work, we study this challenging issue, called support dilution, our goal is to recognize, select, preserve, and enhance those high-contributed supports in the raw support pool. Technically, our method contains three novel parts. First, we propose a contribution index, to quantitatively estimate if a high-contributed support dilutes. Second, we develop the Symmetric Correlation (SC) module to preserve and enhance the high-contributed support features, minimizing the distraction by the low-contributed features. Third, we design the Support Image Pruning operation, to retrieve a compact and high quality subset by discarding low-contributed supports. We conduct extensive experiments on two FSS benchmarks, COCO-20i and PASCAL-5i, the segmentation results demonstrate the compelling performance of our solution over state-of-the-art FSS approaches. Besides, we apply our solution for online segmentation and real-world segmentation, convincing segmentation results showing the practical ability of our work for real-world demonstrations.