PReMVOS: Proposal-generation, Refinement and Merging for Video Object Segmentation

TL;DR

PReMVOS is a semi-supervised video object segmentation method that generates proposals and refines them into consistent object tracks, outperforming previous methods on major benchmarks.

Contribution

The paper introduces PReMVOS, a novel approach that combines proposal generation, refinement, and merging to improve multi-object video segmentation accuracy.

Findings

Achieved state-of-the-art results on DAVIS 2017 with 71.6 J & F score.

Won first place in DAVIS 2018 Video Object Segmentation Challenge.

Secured top position in YouTube-VOS large-scale challenge.

Abstract

We address semi-supervised video object segmentation, the task of automatically generating accurate and consistent pixel masks for objects in a video sequence, given the first-frame ground truth annotations. Towards this goal, we present the PReMVOS algorithm (Proposal-generation, Refinement and Merging for Video Object Segmentation). Our method separates this problem into two steps, first generating a set of accurate object segmentation mask proposals for each video frame and then selecting and merging these proposals into accurate and temporally consistent pixel-wise object tracks over a video sequence in a way which is designed to specifically tackle the difficult challenges involved with segmenting multiple objects across a video sequence. Our approach surpasses all previous state-of-the-art results on the DAVIS 2017 video object segmentation benchmark with a J & F mean score of 71.6 on the test-dev dataset, and achieves first place in both the DAVIS 2018 Video Object Segmentation Challenge and the YouTube-VOS 1st Large-scale Video Object Segmentation Challenge.