Part-aware Panoptic Segmentation

TL;DR

This paper introduces Part-aware Panoptic Segmentation (PPS), a unified scene understanding task that combines scene and part parsing, with new annotations, a dedicated evaluation metric, and baseline results on standard datasets.

Contribution

The paper proposes PPS as a novel unified task, provides annotations and a new metric, and establishes baseline results by merging existing methods.

Findings

PPS effectively combines scene and part parsing.

The PartPQ metric enables comprehensive evaluation.

Baseline results demonstrate the feasibility of PPS.

Abstract

In this work, we introduce the new scene understanding task of Part-aware Panoptic Segmentation (PPS), which aims to understand a scene at multiple levels of abstraction, and unifies the tasks of scene parsing and part parsing. For this novel task, we provide consistent annotations on two commonly used datasets: Cityscapes and Pascal VOC. Moreover, we present a single metric to evaluate PPS, called Part-aware Panoptic Quality (PartPQ). For this new task, using the metric and annotations, we set multiple baselines by merging results of existing state-of-the-art methods for panoptic segmentation and part segmentation. Finally, we conduct several experiments that evaluate the importance of the different levels of abstraction in this single task.