Point2Mask: Point-supervised Panoptic Segmentation via Optimal Transport

TL;DR

Point2Mask introduces a novel point-supervised panoptic segmentation method that formulates pseudo-mask generation as an optimal transport problem, achieving high-quality results with minimal annotation effort.

Contribution

The paper proposes a new OT-based framework for point-supervised panoptic segmentation, utilizing task-oriented maps and a centroid scheme for accurate pseudo-mask generation.

Findings

Effective on Pascal VOC and COCO datasets

Achieves promising segmentation performance with minimal supervision

Introduces a novel OT formulation for pseudo-mask generation

Abstract

Weakly-supervised image segmentation has recently attracted increasing research attentions, aiming to avoid the expensive pixel-wise labeling. In this paper, we present an effective method, namely Point2Mask, to achieve high-quality panoptic prediction using only a single random point annotation per target for training. Specifically, we formulate the panoptic pseudo-mask generation as an Optimal Transport (OT) problem, where each ground-truth (gt) point label and pixel sample are defined as the label supplier and consumer, respectively. The transportation cost is calculated by the introduced task-oriented maps, which focus on the category-wise and instance-wise differences among the various thing and stuff targets. Furthermore, a centroid-based scheme is proposed to set the accurate unit number for each gt point supplier. Hence, the pseudo-mask generation is converted into finding the optimal transport plan at a globally minimal transportation cost, which can be solved via the Sinkhorn-Knopp Iteration. Experimental results on Pascal VOC and COCO demonstrate the promising performance of our proposed Point2Mask approach to point-supervised panoptic segmentation. Source code is available at: https://github.com/LiWentomng/Point2Mask.