Modeling Human Eye Movements with Neural Networks in a Maze-Solving Task

TL;DR

This paper develops a deep generative model of human eye movements during maze-solving, revealing that humans likely use mental simulation rather than task efficiency to guide their gaze.

Contribution

It introduces a novel differentiable architecture for modeling eye movements and proposes that mental simulation underpins human gaze strategies in spatial tasks.

Findings

Human eye movements are best predicted by models simulating internal object traversal.

Humans prioritize mental simulation over task efficiency in gaze control.

The model provides a new computational theory for human maze-solving strategies.

Abstract

From smoothly pursuing moving objects to rapidly shifting gazes during visual search, humans employ a wide variety of eye movement strategies in different contexts. While eye movements provide a rich window into mental processes, building generative models of eye movements is notoriously difficult, and to date the computational objectives guiding eye movements remain largely a mystery. In this work, we tackled these problems in the context of a canonical spatial planning task, maze-solving. We collected eye movement data from human subjects and built deep generative models of eye movements using a novel differentiable architecture for gaze fixations and gaze shifts. We found that human eye movements are best predicted by a model that is optimized not to perform the task as efficiently as possible but instead to run an internal simulation of an object traversing the maze. This not only provides a generative model of eye movements in this task but also suggests a computational theory for how humans solve the task, namely that humans use mental simulation.