Multi-Cue Structure Preserving MRF for Unconstrained Video Segmentation

TL;DR

This paper introduces a multi-cue Markov Random Field model for unconstrained video segmentation, effectively combining contour superpixels, temporal label likelihood, and global structure to improve object region coherence without supervision.

Contribution

The novel integration of multiple cues within an MRF framework enables more accurate and perceptually consistent video segmentation in unconstrained scenarios.

Findings

Significantly outperforms state-of-the-art algorithms on VSB100 dataset.

Produces segmentation results aligned with human perception.

Effective in extracting coherent object regions without supervision.

Abstract

Video segmentation is a stepping stone to understanding video context. Video segmentation enables one to represent a video by decomposing it into coherent regions which comprise whole or parts of objects. However, the challenge originates from the fact that most of the video segmentation algorithms are based on unsupervised learning due to expensive cost of pixelwise video annotation and intra-class variability within similar unconstrained video classes. We propose a Markov Random Field model for unconstrained video segmentation that relies on tight integration of multiple cues: vertices are defined from contour based superpixels, unary potentials from temporal smooth label likelihood and pairwise potentials from global structure of a video. Multi-cue structure is a breakthrough to extracting coherent object regions for unconstrained videos in absence of supervision. Our experiments on VSB100 dataset show that the proposed model significantly outperforms competing state-of-the-art algorithms. Qualitative analysis illustrates that video segmentation result of the proposed model is consistent with human perception of objects.