# Eye movements during free viewing to maximize scene understanding

## TL;DR

People naturally look at people and objects that help them understand a scene the most when viewing images without a specific task.

## Contribution

Free-viewing eye movements are shown to prioritize scene comprehension over salience or object-based tasks.

## Key findings

- Free-viewing fixations resemble those used when describing scenes, not counting or searching.
- Fixations target people and objects whose removal changes scene interpretation.
- Scene comprehension is enhanced when observers describe scenes while fixating on relevant objects.

## Abstract

What humans look at and do when freely viewing a scene is not well understood. We measure observer eye movements under different instructions while participants view a customized set of image pairs containing small visual alterations that greatly change scene interpretation (Winograd images). We show that free-viewing fixations resemble those of observers describing scenes but differ from those of observers counting or searching for objects. Fixations are more often directed toward people and objects whose removal most alters scene interpretation, rather than toward the most salient or meaningfully judged regions (meaning maps), or objects perceived to be grasped or gazed at. Small image changes that modify scene understanding (Winograd images), but not salience or meaning maps, alter fixation patterns. By instructing observers to describe scenes while fixating on objects either relevant or irrelevant to scene understanding, we demonstrate that free-viewing eye movements are functionally important for accurate scene comprehension. Thus, an important human default task of free viewing eye movements is to comprehend scenes.

People often look around without a specific task in mind (free viewing). Here, the authors demonstrate that free viewing eye movements aim to understand scenes by frequently focusing on people and objects that maximize scene comprehension.