Joint Object Contour Points and Semantics for Instance Segmentation

TL;DR

This paper introduces Mask Point R-CNN, a novel instance segmentation method that emphasizes object boundary information by extending keypoint detection to contour points, improving edge sensitivity and segmentation accuracy.

Contribution

The paper proposes a contour point detection auxiliary task for Mask R-CNN, enhancing boundary awareness and geometric feature capture through multi-task training and feature fusion.

Findings

Outperforms vanilla Mask R-CNN by 3.8% on Cityscapes

Achieves 0.8% improvement on COCO

Enhances edge sensitivity and geometric feature extraction

Abstract

The attributes of object contours has great significance for instance segmentation task. However, most of the current popular deep neural networks do not pay much attention to the object edge information. Inspired by the human annotation process when making instance segmentation datasets, in this paper, we propose Mask Point R-CNN aiming at promoting the neural network's attention to the object boundary. Specifically, we innovatively extend the original human keypoint detection task to the contour point detection of any object. Based on this analogy, we present an contour point detection auxiliary task to Mask R-CNN, which can boost the gradient flow between different tasks by effectively using feature fusion strategies and multi-task joint training. As a consequence, the model will be more sensitive to the edges of the object and can capture more geometric features. Quantitatively, the experimental results show that our approach outperforms vanilla Mask R-CNN by 3.8\% on Cityscapes dataset and 0.8\% on COCO dataset.