Pointly-Supervised Panoptic Segmentation

TL;DR

This paper introduces a novel weakly-supervised panoptic segmentation method using only point-level annotations, significantly reducing labeling effort while achieving state-of-the-art results on standard datasets.

Contribution

It presents an end-to-end framework that generates panoptic pseudo-masks from point labels by minimizing pixel-to-point costs, combining semantic, texture, and manifold cues.

Findings

Achieves state-of-the-art performance on Pascal VOC and MS COCO datasets.

Effectively reduces annotation effort compared to fully supervised methods.

Demonstrates the viability of point-level supervision for panoptic segmentation.

Abstract

In this paper, we propose a new approach to applying point-level annotations for weakly-supervised panoptic segmentation. Instead of the dense pixel-level labels used by fully supervised methods, point-level labels only provide a single point for each target as supervision, significantly reducing the annotation burden. We formulate the problem in an end-to-end framework by simultaneously generating panoptic pseudo-masks from point-level labels and learning from them. To tackle the core challenge, i.e., panoptic pseudo-mask generation, we propose a principled approach to parsing pixels by minimizing pixel-to-point traversing costs, which model semantic similarity, low-level texture cues, and high-level manifold knowledge to discriminate panoptic targets. We conduct experiments on the Pascal VOC and the MS COCO datasets to demonstrate the approach's effectiveness and show state-of-the-art performance in the weakly-supervised panoptic segmentation problem. Codes are available at https://github.com/BraveGroup/PSPS.git.