SE-PSNet: Silhouette-based Enhancement Feature for Panoptic Segmentation Network

TL;DR

SE-PSNet introduces a silhouette-based enhancement feature and a new confidence score to improve mask quality and occlusion handling in panoptic segmentation, achieving competitive results with fast inference on COCO and CityScapes datasets.

Contribution

The paper proposes a novel silhouette enhancement feature and confidence score for better mask quality and occlusion handling in panoptic segmentation.

Findings

Achieved competitive results on COCO and CityScapes datasets.

Maintained fast inference speed while improving mask quality.

Effectively handled occlusion issues with the new confidence score.

Abstract

Recently, there has been a panoptic segmentation task combining semantic and instance segmentation, in which the goal is to classify each pixel with the corresponding instance ID. In this work, we propose a solution to tackle the panoptic segmentation task. The overall structure combines the bottom-up method and the top-down method. Therefore, not only can there be better performance, but also the execution speed can be maintained. The network mainly pays attention to the quality of the mask. In the previous work, we can see that the uneven contour of the object is more likely to appear, resulting in low-quality prediction. Accordingly, we propose enhancement features and corresponding loss functions for the silhouette of objects and backgrounds to improve the mask. Meanwhile, we use the new proposed confidence score to solve the occlusion problem and make the network tend to use higher quality masks as prediction results. To verify our research, we used the COCO dataset and CityScapes dataset to do experiments and obtained competitive results with fast inference time.