Treating Motion as Option with Output Selection for Unsupervised Video Object Segmentation

TL;DR

This paper introduces a novel approach for unsupervised video object segmentation that treats motion cues as optional, improving robustness by reducing over-reliance on motion during training and adaptively selecting the best output during testing.

Contribution

The paper proposes a motion-as-option network with adaptive output selection, enabling flexible use of motion cues and enhancing segmentation stability without external guidance.

Findings

Reduces dependency on motion cues during training.

Improves segmentation stability in challenging scenarios.

Achieves better performance with adaptive output selection.

Abstract

Unsupervised video object segmentation aims to detect the most salient object in a video without any external guidance regarding the object. Salient objects often exhibit distinctive movements compared to the background, and recent methods leverage this by combining motion cues from optical flow maps with appearance cues from RGB images. However, because optical flow maps are often closely correlated with segmentation masks, networks can become overly dependent on motion cues during training, leading to vulnerability when faced with confusing motion cues and resulting in unstable predictions. To address this challenge, we propose a novel motion-as-option network that treats motion cues as an optional component rather than a necessity. During training, we randomly input RGB images into the motion encoder instead of optical flow maps, which implicitly reduces the network's reliance on motion cues. This design ensures that the motion encoder is capable of processing both RGB images and optical flow maps, leading to two distinct predictions depending on the type of input provided. To make the most of this flexibility, we introduce an adaptive output selection algorithm that determines the optimal prediction during testing.