Human Scanpath Prediction in Target-Present Visual Search with Semantic-Foveal Bayesian Attention

TL;DR

This paper introduces SemBA-FAST, a novel top-down Bayesian attention model that predicts human visual scanpaths during target-present search tasks by integrating deep object detection with semantic fusion, outperforming existing models.

Contribution

The work presents SemBA-FAST, a new semantic-foveal Bayesian framework that improves human scanpath prediction accuracy in visual search tasks using probabilistic semantic integration.

Findings

SemBA-FAST closely matches human scanpaths on COCO-Search18.

It outperforms baseline and other top-down models.

In some cases, it rivals scanpath-informed models.

Abstract

In goal-directed visual tasks, human perception is guided by both top-down and bottom-up cues. At the same time, foveal vision plays a crucial role in directing attention efficiently. Modern research on bio-inspired computational attention models has taken advantage of advancements in deep learning by utilizing human scanpath data to achieve new state-of-the-art performance. In this work, we assess the performance of SemBA-FAST, i.e. Semantic-based Bayesian Attention for Foveal Active visual Search Tasks, a top-down framework designed for predicting human visual attention in target-present visual search. SemBA-FAST integrates deep object detection with a probabilistic semantic fusion mechanism to generate attention maps dynamically, leveraging pre-trained detectors and artificial foveation to update top-down knowledge and improve fixation prediction sequentially. We evaluate SemBA-FAST on the COCO-Search18 benchmark dataset, comparing its performance against other scanpath prediction models. Our methodology achieves fixation sequences that closely match human ground-truth scanpaths. Notably, it surpasses baseline and other top-down approaches and competes, in some cases, with scanpath-informed models. These findings provide valuable insights into the capabilities of semantic-foveal probabilistic frameworks for human-like attention modelling, with implications for real-time cognitive computing and robotics.