RefineMask: Towards High-Quality Instance Segmentation with Fine-Grained Features

TL;DR

RefineMask introduces a multi-stage approach that incorporates fine-grained features to significantly improve the quality of instance segmentation masks, especially for challenging cases, with minimal additional computational cost.

Contribution

The paper proposes RefineMask, a novel multi-stage method that refines segmentation masks by fusing detailed features, achieving state-of-the-art results on multiple benchmarks.

Findings

Achieves 2.6-3.8 AP improvements over Mask R-CNN on COCO, LVIS, and Cityscapes.

Outperforms the LVIS Challenge 2020 winner with a single model.

Produces high-quality masks with accurate boundaries for difficult object parts.

Abstract

The two-stage methods for instance segmentation, e.g. Mask R-CNN, have achieved excellent performance recently. However, the segmented masks are still very coarse due to the downsampling operations in both the feature pyramid and the instance-wise pooling process, especially for large objects. In this work, we propose a new method called RefineMask for high-quality instance segmentation of objects and scenes, which incorporates fine-grained features during the instance-wise segmenting process in a multi-stage manner. Through fusing more detailed information stage by stage, RefineMask is able to refine high-quality masks consistently. RefineMask succeeds in segmenting hard cases such as bent parts of objects that are over-smoothed by most previous methods and outputs accurate boundaries. Without bells and whistles, RefineMask yields significant gains of 2.6, 3.4, 3.8 AP over Mask R-CNN on COCO, LVIS, and Cityscapes benchmarks respectively at a small amount of additional computational cost. Furthermore, our single-model result outperforms the winner of the LVIS Challenge 2020 by 1.3 points on the LVIS test-dev set and establishes a new state-of-the-art. Code will be available at https://github.com/zhanggang001/RefineMask.