Commonality-Parsing Network across Shape and Appearance for Partially Supervised Instance Segmentation

TL;DR

This paper introduces a novel commonality-parsing network that leverages shape and appearance features to improve partially supervised instance segmentation, enabling better generalization to new categories with limited annotations.

Contribution

It proposes a class-agnostic approach that learns shape and appearance commonalities, outperforming existing methods in partially supervised and few-shot segmentation tasks.

Findings

Significantly outperforms state-of-the-art methods on COCO dataset.

Effectively generalizes to novel categories with limited mask annotations.

Combines shape boundary prediction with pixel affinity modeling.

Abstract

Partially supervised instance segmentation aims to perform learning on limited mask-annotated categories of data thus eliminating expensive and exhaustive mask annotation. The learned models are expected to be generalizable to novel categories. Existing methods either learn a transfer function from detection to segmentation, or cluster shape priors for segmenting novel categories. We propose to learn the underlying class-agnostic commonalities that can be generalized from mask-annotated categories to novel categories. Specifically, we parse two types of commonalities: 1) shape commonalities which are learned by performing supervised learning on instance boundary prediction; and 2) appearance commonalities which are captured by modeling pairwise affinities among pixels of feature maps to optimize the separability between instance and the background. Incorporating both the shape and appearance commonalities, our model significantly outperforms the state-of-the-art methods on both partially supervised setting and few-shot setting for instance segmentation on COCO dataset.