Modeling human visual search: A combined Bayesian searcher and saliency map approach for eye movement guidance in natural scenes

TL;DR

This paper introduces a unified Bayesian model that combines saliency maps with top-down information to predict human eye movement sequences during natural scene visual search tasks, improving over static saliency models.

Contribution

The study presents a novel Bayesian approach that integrates saliency maps as priors, effectively modeling the entire sequence of eye movements in natural scene search tasks.

Findings

Saliency models predict initial fixations well but fail later in the sequence.

The Bayesian model reproduces human scanpaths accurately.

Performance matches human behavior in target detection and scanpath similarity.

Abstract

Finding objects is essential for almost any daily-life visual task. Saliency models have been useful to predict fixation locations in natural images, but are static, i.e., they provide no information about the time-sequence of fixations. Nowadays, one of the biggest challenges in the field is to go beyond saliency maps to predict a sequence of fixations related to a visual task, such as searching for a given target. Bayesian observer models have been proposed for this task, as they represent visual search as an active sampling process. Nevertheless, they were mostly evaluated on artificial images, and how they adapt to natural images remains largely unexplored. Here, we propose a unified Bayesian model for visual search guided by saliency maps as prior information. We validated our model with a visual search experiment in natural scenes recording eye movements. We show that, although state-of-the-art saliency models perform well in predicting the first two fixations in a visual search task, their performance degrades to chance afterward. This suggests that saliency maps alone are good to model bottom-up first impressions, but are not enough to explain the scanpaths when top-down task information is critical. Thus, we propose to use them as priors of Bayesian searchers. This approach leads to a behavior very similar to humans for the whole scanpath, both in the percentage of target found as a function of the fixation rank and the scanpath similarity, reproducing the entire sequence of eye movements.