Dynamical optical flow of saliency maps for predicting visual attention

TL;DR

This paper introduces a mathematically rigorous method for computing saliency maps in dynamic scenes by integrating static saliency maps with optical flow, effectively explaining human attention behavior in complex visual scenarios.

Contribution

The paper presents a novel approach that combines static saliency maps with optical flow to improve dynamic scene saliency prediction, overcoming limitations of existing methods.

Findings

Model accurately predicts human fixation behavior in occlusion scenarios.

Quantitative comparison shows superior performance over alternative models.

Approach explains attention shifts in complex dynamic scenes.

Abstract

Saliency maps are used to understand human attention and visual fixation. However, while very well established for static images, there is no general agreement on how to compute a saliency map of dynamic scenes. In this paper we propose a mathematically rigorous approach to this prob- lem, including static saliency maps of each video frame for the calculation of the optical flow. Taking into account static saliency maps for calculating the optical flow allows for overcoming the aperture problem. Our ap- proach is able to explain human fixation behavior in situations which pose challenges to standard approaches, such as when a fixated object disappears behind an occlusion and reappears after several frames. In addition, we quantitatively compare our model against alternative solutions using a large eye tracking data set. Together, our results suggest that assessing optical flow information across a series of saliency maps gives a highly accurate and useful account of human overt attention in dynamic scenes.