Benchmarking human visual search computational models in natural scenes: models comparison and reference datasets

TL;DR

This paper evaluates and compares state-of-the-art human visual search models in natural scenes using standardized datasets and metrics, proposing an improved hybrid model and emphasizing the need for benchmarking resources.

Contribution

It provides a comprehensive benchmarking framework for visual search models, introduces an improved hybrid model, and highlights the importance of standardized datasets and metrics.

Findings

Current models show limitations in generalization.

Combining Bayesian and neural network approaches improves performance.

Benchmarking datasets are crucial for advancing models.

Abstract

Visual search is an essential part of almost any everyday human goal-directed interaction with the environment. Nowadays, several algorithms are able to predict gaze positions during simple observation, but few models attempt to simulate human behavior during visual search in natural scenes. Furthermore, these models vary widely in their design and exhibit differences in the datasets and metrics with which they were evaluated. Thus, there is a need for a reference point, on which each model can be tested and from where potential improvements can be derived. In the present work, we select publicly available state-of-the-art visual search models in natural scenes and evaluate them on different datasets, employing the same metrics to estimate their efficiency and similarity with human subjects. In particular, we propose an improvement to the Ideal Bayesian Searcher through a combination with a neural network-based visual search model, enabling it to generalize to other datasets. The present work sheds light on the limitations of current models and how potential improvements can be accomplished by combining approaches. Moreover, it moves forward on providing a solution for the urgent need for benchmarking data and metrics to support the development of more general human visual search computational models.