Learning Panoptic Segmentation from Instance Contours

TL;DR

This paper introduces a novel fully convolutional neural network that learns panoptic segmentation by combining semantic segmentation with instance contours, simplifying the process and improving boundary awareness.

Contribution

It proposes a new method that learns instance segmentation directly from semantic segmentation and contours, avoiding separate detection or clustering steps.

Findings

Effective boundary-aware segmentation of objects.

Competitive performance on CityScapes dataset.

Simplified pipeline for panoptic segmentation.

Abstract

Panoptic Segmentation aims to provide an understanding of background (stuff) and instances of objects (things) at a pixel level. It combines the separate tasks of semantic segmentation (pixel level classification) and instance segmentation to build a single unified scene understanding task. Typically, panoptic segmentation is derived by combining semantic and instance segmentation tasks that are learned separately or jointly (multi-task networks). In general, instance segmentation networks are built by adding a foreground mask estimation layer on top of object detectors or using instance clustering methods that assign a pixel to an instance center. In this work, we present a fully convolution neural network that learns instance segmentation from semantic segmentation and instance contours (boundaries of things). Instance contours along with semantic segmentation yield a boundary aware semantic segmentation of things. Connected component labeling on these results produces instance segmentation. We merge semantic and instance segmentation results to output panoptic segmentation. We evaluate our proposed method on the CityScapes dataset to demonstrate qualitative and quantitative performances along with several ablation studies. Our overview video can be accessed from url:https://youtu.be/wBtcxRhG3e0.