Anchor Diffusion for Unsupervised Video Object Segmentation

TL;DR

This paper introduces Anchor Diffusion, a simple method for unsupervised video object segmentation that models long-term dependencies via dense pixel correspondence to improve accuracy and consistency over time.

Contribution

The paper proposes a novel anchor-based diffusion technique to model long-term pixel dependencies without online supervision, outperforming existing methods on key benchmarks.

Findings

Achieves 81.7% mean IoU on DAVIS-2016, ranking first among unsupervised methods.

Outperforms traditional recurrent and optical flow-based approaches.

Demonstrates competitive results on FBMS and ViSal datasets.

Abstract

Unsupervised video object segmentation has often been tackled by methods based on recurrent neural networks and optical flow. Despite their complexity, these kinds of approaches tend to favour short-term temporal dependencies and are thus prone to accumulating inaccuracies, which cause drift over time. Moreover, simple (static) image segmentation models, alone, can perform competitively against these methods, which further suggests that the way temporal dependencies are modelled should be reconsidered. Motivated by these observations, in this paper we explore simple yet effective strategies to model long-term temporal dependencies. Inspired by the non-local operators of [70], we introduce a technique to establish dense correspondences between pixel embeddings of a reference "anchor" frame and the current one. This allows the learning of pairwise dependencies at arbitrarily long distances without conditioning on intermediate frames. Without online supervision, our approach can suppress the background and precisely segment the foreground object even in challenging scenarios, while maintaining consistent performance over time. With a mean IoU of $81.7\%$, our method ranks first on the DAVIS-2016 leaderboard of unsupervised methods, while still being competitive against state-of-the-art online semi-supervised approaches. We further evaluate our method on the FBMS dataset and the ViSal video saliency dataset, showing results competitive with the state of the art.