Real-time Instance Segmentation with Discriminative Orientation Maps

TL;DR

OrienMask is a real-time instance segmentation framework that leverages discriminative orientation maps to efficiently produce masks without extra foreground segmentation, achieving high accuracy and speed on COCO.

Contribution

This paper introduces a novel orientation map-based mask prediction method integrated with YOLOv3 for real-time instance segmentation.

Findings

Achieves 34.8 mask AP on COCO

Runs at 42.7 fps on a single RTX 2080 Ti

Maintains competitive accuracy with a concise mask representation

Abstract

Although instance segmentation has made considerable advancement over recent years, it's still a challenge to design high accuracy algorithms with real-time performance. In this paper, we propose a real-time instance segmentation framework termed OrienMask. Upon the one-stage object detector YOLOv3, a mask head is added to predict some discriminative orientation maps, which are explicitly defined as spatial offset vectors for both foreground and background pixels. Thanks to the discrimination ability of orientation maps, masks can be recovered without the need for extra foreground segmentation. All instances that match with the same anchor size share a common orientation map. This special sharing strategy reduces the amortized memory utilization for mask predictions but without loss of mask granularity. Given the surviving box predictions after NMS, instance masks can be concurrently constructed from the corresponding orientation maps with low complexity. Owing to the concise design for mask representation and its effective integration with the anchor-based object detector, our method is qualified under real-time conditions while maintaining competitive accuracy. Experiments on COCO benchmark show that OrienMask achieves 34.8 mask AP at the speed of 42.7 fps evaluated with a single RTX 2080 Ti. The code is available at https://github.com/duwt/OrienMask.