CT-ScanGaze: A Dataset and Baselines for 3D Volumetric Scanpath Modeling

TL;DR

This paper introduces CT-ScanGaze, the first publicly available 3D eye gaze dataset for CT scans, and proposes CT-Searcher, a novel model for predicting radiologist-like 3D scanpaths, advancing interpretability in medical imaging.

Contribution

The paper provides a new 3D eye gaze dataset for CT scans and develops a specialized 3D scanpath predictor with a pretraining pipeline from 2D gaze data.

Findings

CT-ScanGaze enables new research in 3D gaze analysis.

CT-Searcher outperforms existing 2D models in 3D scanpath prediction.

Pretraining with converted 2D data improves model performance.

Abstract

Understanding radiologists' eye movement during Computed Tomography (CT) reading is crucial for developing effective interpretable computer-aided diagnosis systems. However, CT research in this area has been limited by the lack of publicly available eye-tracking datasets and the three-dimensional complexity of CT volumes. To address these challenges, we present the first publicly available eye gaze dataset on CT, called CT-ScanGaze. Then, we introduce CT-Searcher, a novel 3D scanpath predictor designed specifically to process CT volumes and generate radiologist-like 3D fixation sequences, overcoming the limitations of current scanpath predictors that only handle 2D inputs. Since deep learning models benefit from a pretraining step, we develop a pipeline that converts existing 2D gaze datasets into 3D gaze data to pretrain CT-Searcher. Through both qualitative and quantitative evaluations on CT-ScanGaze, we demonstrate the effectiveness of our approach and provide a comprehensive assessment framework for 3D scanpath prediction in medical imaging.