Feature Weighting and Boosting for Few-Shot Segmentation

TL;DR

This paper introduces a novel few-shot segmentation method that enhances feature discriminativeness and employs ensemble boosting, leading to significant performance improvements on standard datasets.

Contribution

It proposes two key innovations: improving feature discriminativeness for better segmentation and boosting inference with an ensemble guided by loss gradients.

Findings

Outperforms existing methods on PASCAL-5i dataset

Outperforms existing methods on COCO-20i dataset

Demonstrates significant accuracy improvements

Abstract

This paper is about few-shot segmentation of foreground objects in images. We train a CNN on small subsets of training images, each mimicking the few-shot setting. In each subset, one image serves as the query and the other(s) as support image(s) with ground-truth segmentation. The CNN first extracts feature maps from the query and support images. Then, a class feature vector is computed as an average of the support's feature maps over the known foreground. Finally, the target object is segmented in the query image by using a cosine similarity between the class feature vector and the query's feature map. We make two contributions by: (1) Improving discriminativeness of features so their activations are high on the foreground and low elsewhere; and (2) Boosting inference with an ensemble of experts guided with the gradient of loss incurred when segmenting the support images in testing. Our evaluations on the PASCAL-$5^i$ and COCO-$20^i$ datasets demonstrate that we significantly outperform existing approaches.