Graph-Theoretic Spatiotemporal Context Modeling for Video Saliency Detection

TL;DR

This paper introduces a graph-theoretic method for video saliency detection that models spatiotemporal interactions among atomic video structures to improve detection accuracy and consistency.

Contribution

It presents a novel approach using adaptive video structure discovery within a spatiotemporal graph to capture contextual interactions for saliency detection.

Findings

Effective modeling of semantic contextual interactions

Preserves spatial smoothness and temporal consistency

Outperforms existing methods on benchmark datasets

Abstract

As an important and challenging problem in computer vision, video saliency detection is typically cast as a spatiotemporal context modeling problem over consecutive frames. As a result, a key issue in video saliency detection is how to effectively capture the intrinsical properties of atomic video structures as well as their associated contextual interactions along the spatial and temporal dimensions. Motivated by this observation, we propose a graph-theoretic video saliency detection approach based on adaptive video structure discovery, which is carried out within a spatiotemporal atomic graph. Through graph-based manifold propagation, the proposed approach is capable of effectively modeling the semantically contextual interactions among atomic video structures for saliency detection while preserving spatial smoothness and temporal consistency. Experiments demonstrate the effectiveness of the proposed approach over several benchmark datasets.