Task-Adaptive Feature Transformer with Semantic Enrichment for Few-Shot Segmentation

TL;DR

This paper introduces a task-adaptive feature transformer and semantic enrichment modules that enhance existing segmentation networks for few-shot segmentation, achieving competitive results on standard datasets.

Contribution

The paper proposes a novel learnable module, TAFT, for task-specific feature transformation, and a semantic enrichment module utilizing pixel-wise attention and auxiliary loss, improving few-shot segmentation performance.

Findings

Achieves state-of-the-art results on PASCAL-5^i and COCO-20^i datasets.

Effectively utilizes semantic information for better segmentation masks.

Extends existing segmentation networks with minimal additional modules.

Abstract

Few-shot learning allows machines to classify novel classes using only a few labeled samples. Recently, few-shot segmentation aiming at semantic segmentation on low sample data has also seen great interest. In this paper, we propose a learnable module that can be placed on top of existing segmentation networks for performing few-shot segmentation. This module, called the task-adaptive feature transformer (TAFT), linearly transforms task-specific high-level features to a set of task agnostic features well-suited to conducting few-shot segmentation. The task-conditioned feature transformation allows an effective utilization of the semantic information in novel classes to generate tight segmentation masks. We also propose a semantic enrichment (SE) module that utilizes a pixel-wise attention module for high-level feature and an auxiliary loss from an auxiliary segmentation network conducting the semantic segmentation for all training classes. Experiments on PASCAL-$5^i$ and COCO-$20^i$ datasets confirm that the added modules successfully extend the capability of existing segmentators to yield highly competitive few-shot segmentation performances.