SUNet: Scale-aware Unified Network for Panoptic Segmentation

TL;DR

SUNet is a novel panoptic segmentation network that effectively handles multi-scale objects by integrating global context modeling and high-resolution feature collection, improving segmentation accuracy for both large and small objects.

Contribution

The paper introduces two lightweight modules, Pixel-relation Block and Convectional Network, to enhance scale-awareness in panoptic segmentation within a unified end-to-end framework.

Findings

Improved segmentation accuracy on Cityscapes and COCO datasets.

Effective handling of objects at various scales with lightweight modules.

Demonstrated adaptability and efficiency of SUNet in multi-scale environments.

Abstract

Panoptic segmentation combines the advantages of semantic and instance segmentation, which can provide both pixel-level and instance-level environmental perception information for intelligent vehicles. However, it is challenged with segmenting objects of various scales, especially on extremely large and small ones. In this work, we propose two lightweight modules to mitigate this problem. First, Pixel-relation Block is designed to model global context information for large-scale things, which is based on a query-independent formulation and brings small parameter increments. Then, Convectional Network is constructed to collect extra high-resolution information for small-scale stuff, supplying more appropriate semantic features for the downstream segmentation branches. Based on these two modules, we present an end-to-end Scale-aware Unified Network (SUNet), which is more adaptable to multi-scale objects. Extensive experiments on Cityscapes and COCO demonstrate the effectiveness of the proposed methods.