Simulating Human Gaze with Neural Visual Attention

TL;DR

This paper introduces the Neural Visual Attention (NeVA) algorithm, which models human-like gaze patterns by integrating task guidance and biological constraints into neural networks, outperforming existing models on benchmark datasets.

Contribution

The paper presents NeVA, a novel attention mechanism that incorporates biological foveated vision constraints and task guidance, enabling more realistic human-like gaze simulations.

Findings

NeVA generates human-like scanpaths without explicit training for this.

It outperforms state-of-the-art unsupervised models on benchmark datasets.

Biologically constrained networks naturally produce human-like visual explorations.

Abstract

Existing models of human visual attention are generally unable to incorporate direct task guidance and therefore cannot model an intent or goal when exploring a scene. To integrate guidance of any downstream visual task into attention modeling, we propose the Neural Visual Attention (NeVA) algorithm. To this end, we impose to neural networks the biological constraint of foveated vision and train an attention mechanism to generate visual explorations that maximize the performance with respect to the downstream task. We observe that biologically constrained neural networks generate human-like scanpaths without being trained for this objective. Extensive experiments on three common benchmark datasets show that our method outperforms state-of-the-art unsupervised human attention models in generating human-like scanpaths.