A Simple and Powerful Global Optimization for Unsupervised Video Object Segmentation

TL;DR

This paper introduces a simple, scalable, and dataset-agnostic method for unsupervised video object segmentation that leverages spectral clustering on features and optical flows, achieving competitive results with less complexity.

Contribution

It presents a novel objective function for unsupervised video segmentation derived from spectral clustering, requiring only off-the-shelf features and flows, and scales efficiently with sequence length.

Findings

Achieves state-of-the-art performance on standard benchmarks.

Simpler and more scalable than existing methods.

Generalizes well across different datasets.

Abstract

We propose a simple, yet powerful approach for unsupervised object segmentation in videos. We introduce an objective function whose minimum represents the mask of the main salient object over the input sequence. It only relies on independent image features and optical flows, which can be obtained using off-the-shelf self-supervised methods. It scales with the length of the sequence with no need for superpixels or sparsification, and it generalizes to different datasets without any specific training. This objective function can actually be derived from a form of spectral clustering applied to the entire video. Our method achieves on-par performance with the state of the art on standard benchmarks (DAVIS2016, SegTrack-v2, FBMS59), while being conceptually and practically much simpler. Code is available at https://ponimatkin.github.io/ssl-vos.