Prior to Segment: Foreground Cues for Weakly Annotated Classes in Partially Supervised Instance Segmentation

TL;DR

This paper introduces an object mask prior (OMP) to enhance weakly supervised instance segmentation by providing a general foreground concept, significantly improving performance over baseline models.

Contribution

The paper proposes an object mask prior (OMP) that helps class-agnostic mask heads learn a general foreground concept using box supervision, improving weakly supervised segmentation.

Findings

OMP significantly improves mask prediction accuracy.

Approach achieves competitive results with state-of-the-art methods.

Method simplifies architecture while enhancing performance.

Abstract

Instance segmentation methods require large datasets with expensive and thus limited instance-level mask labels. Partially supervised instance segmentation aims to improve mask prediction with limited mask labels by utilizing the more abundant weak box labels. In this work, we show that a class agnostic mask head, commonly used in partially supervised instance segmentation, has difficulties learning a general concept of foreground for the weakly annotated classes using box supervision only. To resolve this problem we introduce an object mask prior (OMP) that provides the mask head with the general concept of foreground implicitly learned by the box classification head under the supervision of all classes. This helps the class agnostic mask head to focus on the primary object in a region of interest (RoI) and improves generalization to the weakly annotated classes. We test our approach on the COCO dataset using different splits of strongly and weakly supervised classes. Our approach significantly improves over the Mask R-CNN baseline and obtains competitive performance with the state-of-the-art, while offering a much simpler architecture.