CRNet: Cross-Reference Networks for Few-Shot Segmentation

TL;DR

CRNet introduces a cross-reference mechanism and mask refinement for few-shot segmentation, enabling better co-occurrent object detection and improved performance with limited support images.

Contribution

The paper proposes CRNet, a novel few-shot segmentation model that predicts masks for both support and query images simultaneously using cross-reference, and refines foreground predictions iteratively.

Findings

Achieves state-of-the-art results on PASCAL VOC 2012.

Effectively utilizes multiple support images with fine-tuning.

Improves co-occurrent object detection in few-shot segmentation.

Abstract

Over the past few years, state-of-the-art image segmentation algorithms are based on deep convolutional neural networks. To render a deep network with the ability to understand a concept, humans need to collect a large amount of pixel-level annotated data to train the models, which is time-consuming and tedious. Recently, few-shot segmentation is proposed to solve this problem. Few-shot segmentation aims to learn a segmentation model that can be generalized to novel classes with only a few training images. In this paper, we propose a cross-reference network (CRNet) for few-shot segmentation. Unlike previous works which only predict the mask in the query image, our proposed model concurrently make predictions for both the support image and the query image. With a cross-reference mechanism, our network can better find the co-occurrent objects in the two images, thus helping the few-shot segmentation task. We also develop a mask refinement module to recurrently refine the prediction of the foreground regions. For the $k$-shot learning, we propose to finetune parts of networks to take advantage of multiple labeled support images. Experiments on the PASCAL VOC 2012 dataset show that our network achieves state-of-the-art performance.