A New Few-shot Segmentation Network Based on Class Representation

TL;DR

This paper introduces a novel few-shot segmentation network that represents unseen classes through existing class knowledge, improving segmentation accuracy on Pascal VOC 2012 with a new class activation map generation approach.

Contribution

It proposes a class representation-based framework and a CAM generation module for better unseen class modeling in few-shot segmentation.

Findings

Achieves 69.2% FB-IoU in one-shot setting

Achieves 70.1% FB-IoU in five-shot setting

Outperforms state-of-the-art methods on Pascal VOC 2012

Abstract

This paper studies few-shot segmentation, which is a task of predicting foreground mask of unseen classes by a few of annotations only, aided by a set of rich annotations already existed. The existing methods mainly focus the task on "\textit{how to transfer segmentation cues from support images (labeled images) to query images (unlabeled images)}", and try to learn efficient and general transfer module that can be easily extended to unseen classes. However, it is proved to be a challenging task to learn the transfer module that is general to various classes. This paper solves few-shot segmentation in a new perspective of "\textit{how to represent unseen classes by existing classes}", and formulates few-shot segmentation as the representation process that represents unseen classes (in terms of forming the foreground prior) by existing classes precisely. Based on such idea, we propose a new class representation based few-shot segmentation framework, which firstly generates class activation map of unseen class based on the knowledge of existing classes, and then uses the map as foreground probability map to extract the foregrounds from query image. A new two-branch based few-shot segmentation network is proposed. Moreover, a new CAM generation module that extracts the CAM of unseen classes rather than the classical training classes is raised. We validate the effectiveness of our method on Pascal VOC 2012 dataset, the value FB-IoU of one-shot and five-shot arrives at 69.2\% and 70.1\% respectively, which outperforms the state-of-the-art method.