Maps of Visual Importance

TL;DR

This paper introduces a novel method to identify visually important elements in images by analyzing eye movements during image recall, revealing that certain fixations relate to memory rather than attention.

Contribution

It proposes a deformation mapping technique to filter fixations based on recall exploration, offering a new way to measure visual importance beyond attention-based fixations.

Findings

Certain scene elements are filtered out, indicating they are less important.

Recall fixations are only qualitatively related to exploration fixations.

The method provides a controllable filter for visual importance analysis.

Abstract

The importance of an element in a visual stimulus is commonly associated with the fixations during a free-viewing task. We argue that fixations are not always correlated with attention or awareness of visual objects. We suggest to filter the fixations recorded during exploration of the image based on the fixations recorded during recalling the image against a neutral background. This idea exploits that eye movements are a spatial index into the memory of a visual stimulus. We perform an experiment in which we record the eye movements of 30 observers during the presentation and recollection of 100 images. The locations of fixations during recall are only qualitatively related to the fixations during exploration. We develop a deformation mapping technique to align the fixations from recall with the fixation during exploration. This allows filtering the fixations based on proximity and a threshold on proximity provides a convenient slider to control the amount of filtering. Analyzing the spatial histograms resulting from the filtering procedure as well as the set of removed fixations shows that certain types of scene elements, which could be considered irrelevant, are removed. In this sense, they provide a measure of importance of visual elements for human observers.