Video Object Segmentation with Re-identification

TL;DR

This paper introduces VS-ReID, a video object segmentation model that combines mask propagation and adaptive re-identification to improve accuracy and robustness against large displacements and drifting.

Contribution

The paper proposes a novel re-identification mechanism integrated with mask propagation for more reliable video object segmentation.

Findings

Achieves a global mean of 0.699 on DAVIS 2017, outperforming previous methods.

Effectively handles large displacements and target re-identification.

Sets new state-of-the-art performance in video segmentation challenge.

Abstract

Conventional video segmentation methods often rely on temporal continuity to propagate masks. Such an assumption suffers from issues like drifting and inability to handle large displacement. To overcome these issues, we formulate an effective mechanism to prevent the target from being lost via adaptive object re-identification. Specifically, our Video Object Segmentation with Re-identification (VS-ReID) model includes a mask propagation module and a ReID module. The former module produces an initial probability map by flow warping while the latter module retrieves missing instances by adaptive matching. With these two modules iteratively applied, our VS-ReID records a global mean (Region Jaccard and Boundary F measure) of 0.699, the best performance in 2017 DAVIS Challenge.