Foveated Model Observers for Visual Search in 3D Medical Images

TL;DR

This study introduces a foveated model observer that incorporates eye movement and reduced peripheral detail to better predict human performance in 3D medical image search tasks, especially for signals less detectable in the visual periphery.

Contribution

The paper proposes a novel foveated extension of the Channelized Hotelling model that accounts for peripheral vision and eye movements, improving prediction of human performance in 3D search tasks.

Findings

Foveated model predicts human interaction between 3D search and signal detectability.

Standard models fail to capture peripheral detection effects in 3D search.

Foveated model accurately predicts errors and performance patterns in 3D medical image search.

Abstract

Model observers have a long history of success in predicting human observer performance in clinically-relevant detection tasks. New 3D image modalities provide more signal information but vastly increase the search space to be scrutinized. Here, we compared standard linear model observers (ideal observers, non-pre-whitening matched filter with eye filter, and various versions of Channelized Hotelling models) to human performance searching in 3D 1/f$^{2.8}$ filtered noise images and assessed its relationship to the more traditional location known exactly detection tasks and 2D search. We investigated two different signal types that vary in their detectability away from the point of fixation (visual periphery). We show that the influence of 3D search on human performance interacts with the signal's detectability in the visual periphery. Detection performance for signals difficult to detect in the visual periphery deteriorates greatly in 3D search but not in 3D location known exactly and 2D search. Standard model observers do not predict the interaction between 3D search and signal type. A proposed extension of the Channelized Hotelling model (foveated search model) that processes the image with reduced spatial detail away from the point of fixation, explores the image through eye movements, and scrolls across slices can successfully predict the interaction observed in humans and also the types of errors in 3D search. Together, the findings highlight the need for foveated model observers for image quality evaluation with 3D search.