ContourRender: Detecting Arbitrary Contour Shape For Instance Segmentation In One Pass

TL;DR

ContourRender introduces a novel differentiable rendering approach for one-pass arbitrary contour shape detection in instance segmentation, improving contour quality without iterative refinement and maintaining high inference speed.

Contribution

It proposes ContourRender, a shape signature-based, differentiable rendering method that enhances contour prediction accuracy in one pass, surpassing previous contour-based methods.

Findings

Outperforms all contour-based methods on COCO

Competitive with iteration-based state-of-the-art on Cityscapes

Improves contour quality on COCO ContourHard-val

Abstract

Direct contour regression for instance segmentation is a challenging task. Previous works usually achieve it by learning to progressively refine the contour prediction or adopting a shape representation with limited expressiveness. In this work, we argue that the difficulty in regressing the contour points in one pass is mainly due to the ambiguity when discretizing a smooth contour into a polygon. To address the ambiguity, we propose a novel differentiable rendering-based approach named \textbf{ContourRender}. During training, it first predicts a contour generated by an invertible shape signature, and then optimizes the contour with the more stable silhouette by converting it to a contour mesh and rendering the mesh to a 2D map. This method significantly improves the quality of contour without iterations or cascaded refinements. Moreover, as optimization is not needed during inference, the inference speed will not be influenced. Experiments show the proposed ContourRender outperforms all the contour-based instance segmentation approaches on COCO, while stays competitive with the iteration-based state-of-the-art on Cityscapes. In addition, we specifically select a subset from COCO val2017 named COCO ContourHard-val to further demonstrate the contour quality improvements. Codes, models, and dataset split will be released.