Amodal segmentation just like doing a jigsaw

TL;DR

This paper introduces a novel amodal segmentation approach inspired by a jigsaw puzzle, which predicts visible and occluded parts separately and then combines them, effectively modeling occlusion relationships.

Contribution

The proposed method uses a jigsaw-inspired multi-task approach to improve amodal segmentation by better modeling occlusion context and reducing duplicate information.

Findings

Outperforms existing state-of-the-art methods on two datasets

Effectively models occlusion relationships

Utilizes a novel jigsaw-inspired segmentation strategy

Abstract

Amodal segmentation is a new direction of instance segmentation while considering the segmentation of the visible and occluded parts of the instance. The existing state-of-the-art method uses multi-task branches to predict the amodal part and the visible part separately and subtract the visible part from the amodal part to obtain the occluded part. However, the amodal part contains visible information. Therefore, the separated prediction method will generate duplicate information. Different from this method, we propose a method of amodal segmentation based on the idea of the jigsaw. The method uses multi-task branches to predict the two naturally decoupled parts of visible and occluded, which is like getting two matching jigsaw pieces. Then put the two jigsaw pieces together to get the amodal part. This makes each branch focus on the modeling of the object. And we believe that there are certain rules in the occlusion relationship in the real world. This is a kind of occlusion context information. This jigsaw method can better model the occlusion relationship and use the occlusion context information, which is important for amodal segmentation. Experiments on two widely used amodally annotated datasets prove that our method exceeds existing state-of-the-art methods. The source code of this work will be made public soon.