Mask Scoring R-CNN

TL;DR

Mask Scoring R-CNN introduces a network component that learns to predict the quality of instance masks, aligning mask scores with actual mask accuracy, thereby improving overall instance segmentation performance.

Contribution

The paper proposes a novel mask scoring network that predicts mask IoU to better calibrate mask confidence scores in instance segmentation.

Findings

Consistent performance improvements on COCO dataset

Outperforms previous Mask R-CNN models

Effective mask quality calibration enhances segmentation accuracy

Abstract

Letting a deep network be aware of the quality of its own predictions is an interesting yet important problem. In the task of instance segmentation, the confidence of instance classification is used as mask quality score in most instance segmentation frameworks. However, the mask quality, quantified as the IoU between the instance mask and its ground truth, is usually not well correlated with classification score. In this paper, we study this problem and propose Mask Scoring R-CNN which contains a network block to learn the quality of the predicted instance masks. The proposed network block takes the instance feature and the corresponding predicted mask together to regress the mask IoU. The mask scoring strategy calibrates the misalignment between mask quality and mask score, and improves instance segmentation performance by prioritizing more accurate mask predictions during COCO AP evaluation. By extensive evaluations on the COCO dataset, Mask Scoring R-CNN brings consistent and noticeable gain with different models, and outperforms the state-of-the-art Mask R-CNN. We hope our simple and effective approach will provide a new direction for improving instance segmentation. The source code of our method is available at \url{https://github.com/zjhuang22/maskscoring_rcnn}.