CenterPoly: real-time instance segmentation using bounding polygons

TL;DR

CenterPoly is a real-time instance segmentation method that uses bounding polygons and center keypoints, optimized for dense urban scenes and intelligent transportation systems, achieving state-of-the-art speed and accuracy.

Contribution

It introduces a lightweight, parallel detection and segmentation approach with a novel vertex selection strategy and depth estimation for overlapping objects.

Findings

Achieves real-time speed on Cityscapes, KITTI, and IDD datasets.

Outperforms existing methods in accuracy at comparable speeds.

Provides a flexible model with different backbones for speed-accuracy trade-offs.

Abstract

We present a novel method, called CenterPoly, for real-time instance segmentation using bounding polygons. We apply it to detect road users in dense urban environments, making it suitable for applications in intelligent transportation systems like automated vehicles. CenterPoly detects objects by their center keypoint while predicting a fixed number of polygon vertices for each object, thus performing detection and segmentation in parallel. Most of the network parameters are shared by the network heads, making it fast and lightweight enough to run at real-time speed. To properly convert mask ground-truth to polygon ground-truth, we designed a vertex selection strategy to facilitate the learning of the polygons. Additionally, to better segment overlapping objects in dense urban scenes, we also train a relative depth branch to determine which instances are closer and which are further, using available weak annotations. We propose several models with different backbones to show the possible speed / accuracy trade-offs. The models were trained and evaluated on Cityscapes, KITTI and IDD and the results are reported on their public benchmark, which are state-of-the-art at real-time speeds. Code is available at https://github.com/hu64/CenterPoly