Unsupervised Video Object Segmentation with Distractor-Aware Online Adaptation

TL;DR

This paper introduces a novel unsupervised video object segmentation method called Distractor-Aware Online Adaptation (DOA), which models spatial-temporal consistency and effectively handles distractors to improve segmentation accuracy.

Contribution

The paper proposes a new distractor-aware online adaptation approach that captures background dependencies and utilizes hard negatives for improved unsupervised video object segmentation.

Findings

Achieves state-of-the-art results on DAVIS 2016 and FBMS-59 datasets.

Effectively models background dependencies using spatial-temporal consistency.

Utilizes hard negatives and general negatives to enhance segmentation accuracy.

Abstract

Unsupervised video object segmentation is a crucial application in video analysis without knowing any prior information about the objects. It becomes tremendously challenging when multiple objects occur and interact in a given video clip. In this paper, a novel unsupervised video object segmentation approach via distractor-aware online adaptation (DOA) is proposed. DOA models spatial-temporal consistency in video sequences by capturing background dependencies from adjacent frames. Instance proposals are generated by the instance segmentation network for each frame and then selected by motion information as hard negatives if they exist and positives. To adopt high-quality hard negatives, the block matching algorithm is then applied to preceding frames to track the associated hard negatives. General negatives are also introduced in case that there are no hard negatives in the sequence and experiments demonstrate both kinds of negatives (distractors) are complementary. Finally, we conduct DOA using the positive, negative, and hard negative masks to update the foreground/background segmentation. The proposed approach achieves state-of-the-art results on two benchmark datasets, DAVIS 2016 and FBMS-59 datasets.