Unsupervised Video Object Segmentation using Motion Saliency-Guided Spatio-Temporal Propagation

TL;DR

This paper introduces a novel unsupervised video segmentation method that leverages motion saliency and a new neighborhood graph to improve segmentation accuracy, outperforming many existing deep learning approaches in challenging scenarios.

Contribution

The paper presents a new saliency estimation technique and neighborhood graph based on optical flow and edge cues, enhancing unsupervised segmentation performance.

Findings

Achieves state-of-the-art results on DAVIS, SegTrack v2, FBMS-59 datasets.

Outperforms deep learning methods in unsupervised setting.

Demonstrates competitive semi-supervised results with minimal training data.

Abstract

Unsupervised video segmentation plays an important role in a wide variety of applications from object identification to compression. However, to date, fast motion, motion blur and occlusions pose significant challenges. To address these challenges for unsupervised video segmentation, we develop a novel saliency estimation technique as well as a novel neighborhood graph, based on optical flow and edge cues. Our approach leads to significantly better initial foreground-background estimates and their robust as well as accurate diffusion across time. We evaluate our proposed algorithm on the challenging DAVIS, SegTrack v2 and FBMS-59 datasets. Despite the usage of only a standard edge detector trained on 200 images, our method achieves state-of-the-art results outperforming deep learning based methods in the unsupervised setting. We even demonstrate competitive results comparable to deep learning based methods in the semi-supervised setting on the DAVIS dataset.