Integrating Eye-Gaze Data into CXR DL Approaches: A Preliminary study

TL;DR

This study explores the integration of eye-gaze data into deep learning models for chest X-ray abnormality detection, finding that direct application of gaze data does not improve predictive performance, highlighting the need for further research.

Contribution

It introduces a novel multimodal deep learning architecture combining medical images and eye-tracking data for chest X-ray analysis.

Findings

Eye gaze data did not improve abnormality detection accuracy.

Results align with previous literature questioning direct gaze data integration.

Further investigation is needed for effective use of human-generated data in DL models.

Abstract

This paper proposes a novel multimodal DL architecture incorporating medical images and eye-tracking data for abnormality detection in chest x-rays. Our results show that applying eye gaze data directly into DL architectures does not show superior predictive performance in abnormality detection chest X-rays. These results support other works in the literature and suggest that human-generated data, such as eye gaze, needs a more thorough investigation before being applied to DL architectures.