Fast Panoptic Segmentation Network

TL;DR

FPSNet introduces a fast, end-to-end panoptic segmentation network that simplifies the process by pixel-wise classification, achieving high speed and competitive accuracy on standard datasets.

Contribution

The paper proposes FPSNet, a novel panoptic segmentation approach that eliminates the need for instance mask prediction and merging heuristics, enabling real-time performance.

Findings

Achieves a Panoptic Quality score of 55.1% on Cityscapes.

Runs at 114 ms per image at high resolution.

Operates at 22 and 35 fps on Cityscapes and Pascal VOC datasets.

Abstract

In this work, we present an end-to-end network for fast panoptic segmentation. This network, called Fast Panoptic Segmentation Network (FPSNet), does not require computationally costly instance mask predictions or merging heuristics. This is achieved by casting the panoptic task into a custom dense pixel-wise classification task, which assigns a class label or an instance id to each pixel. We evaluate FPSNet on the Cityscapes and Pascal VOC datasets, and find that FPSNet is faster than existing panoptic segmentation methods, while achieving better or similar panoptic segmentation performance. On the Cityscapes validation set, we achieve a Panoptic Quality score of 55.1%, at prediction times of 114 milliseconds for images with a resolution of 1024x2048 pixels. For lower resolutions of the Cityscapes dataset and for the Pascal VOC dataset, FPSNet runs at 22 and 35 frames per second, respectively.