PatchDCT: Patch Refinement for High Quality Instance Segmentation

TL;DR

PatchDCT introduces a novel multi-stage refinement framework for high-quality instance segmentation by dividing masks into patches and refining each with classifiers and regressors, significantly improving accuracy.

Contribution

It proposes PatchDCT, a new method that refines DCT-based masks by patch-wise processing, enhancing segmentation quality over previous DCT-Mask approaches.

Findings

Achieves up to 4.5% AP improvement on COCO

Surpasses DCT-Mask in boundary AP by up to 4.2%

Competitive with state-of-the-art segmentation methods

Abstract

High-quality instance segmentation has shown emerging importance in computer vision. Without any refinement, DCT-Mask directly generates high-resolution masks by compressed vectors. To further refine masks obtained by compressed vectors, we propose for the first time a compressed vector based multi-stage refinement framework. However, the vanilla combination does not bring significant gains, because changes in some elements of the DCT vector will affect the prediction of the entire mask. Thus, we propose a simple and novel method named PatchDCT, which separates the mask decoded from a DCT vector into several patches and refines each patch by the designed classifier and regressor. Specifically, the classifier is used to distinguish mixed patches from all patches, and to correct previously mispredicted foreground and background patches. In contrast, the regressor is used for DCT vector prediction of mixed patches, further refining the segmentation quality at boundary locations. Experiments on COCO show that our method achieves 2.0%, 3.2%, 4.5% AP and 3.4%, 5.3%, 7.0% Boundary AP improvements over Mask-RCNN on COCO, LVIS, and Cityscapes, respectively. It also surpasses DCT-Mask by 0.7%, 1.1%, 1.3% AP and 0.9%, 1.7%, 4.2% Boundary AP on COCO, LVIS and Cityscapes. Besides, the performance of PatchDCT is also competitive with other state-of-the-art methods.