Look Closer to Segment Better: Boundary Patch Refinement for Instance Segmentation

TL;DR

This paper introduces BPR, a boundary patch refinement framework that enhances instance segmentation masks by focusing on boundary areas, significantly improving boundary quality and achieving top results on Cityscapes.

Contribution

The paper presents a simple, effective post-processing method that refines boundary patches in instance segmentation masks, boosting accuracy especially on boundary-aware metrics.

Findings

Significant improvement over Mask R-CNN on Cityscapes

Achieved 1st place on Cityscapes leaderboard with BPR

Enhanced boundary quality in instance segmentation masks

Abstract

Tremendous efforts have been made on instance segmentation but the mask quality is still not satisfactory. The boundaries of predicted instance masks are usually imprecise due to the low spatial resolution of feature maps and the imbalance problem caused by the extremely low proportion of boundary pixels. To address these issues, we propose a conceptually simple yet effective post-processing refinement framework to improve the boundary quality based on the results of any instance segmentation model, termed BPR. Following the idea of looking closer to segment boundaries better, we extract and refine a series of small boundary patches along the predicted instance boundaries. The refinement is accomplished by a boundary patch refinement network at higher resolution. The proposed BPR framework yields significant improvements over the Mask R-CNN baseline on Cityscapes benchmark, especially on the boundary-aware metrics. Moreover, by applying the BPR framework to the PolyTransform + SegFix baseline, we reached 1st place on the Cityscapes leaderboard.