Discrimination of Radiologists Utilizing Eye-Tracking Technology and Machine Learning: A Case Study

TL;DR

This study introduces a novel eye-tracking and machine learning approach to distinguish radiologists' experience levels based on their visual search patterns during chest X-ray interpretation, promising improvements in training and quality control.

Contribution

The paper presents a new discretized feature encoding method for eye-tracking data and demonstrates its effectiveness in classifying radiologists' expertise levels, outperforming existing methods.

Findings

Classifiers with the proposed features outperform state-of-the-art methods.

The approach reliably discriminates radiologists' experience levels.

Robustness confirmed across different datasets and protocols.

Abstract

Perception-related errors comprise most diagnostic mistakes in radiology. To mitigate this problem, radiologists employ personalized and high-dimensional visual search strategies, otherwise known as search patterns. Qualitative descriptions of these search patterns, which involve the physician verbalizing or annotating the order he/she analyzes the image, can be unreliable due to discrepancies in what is reported versus the actual visual patterns. This discrepancy can interfere with quality improvement interventions and negatively impact patient care. This study presents a novel discretized feature encoding based on spatiotemporal binning of fixation data for efficient geometric alignment and temporal ordering of eye movement when reading chest X-rays. The encoded features of the eye-fixation data are employed by machine learning classifiers to discriminate between faculty and trainee radiologists. We include a clinical trial case study utilizing the Area Under the Curve (AUC), Accuracy, F1, Sensitivity, and Specificity metrics for class separability to evaluate the discriminability between the two subjects in regard to their level of experience. We then compare the classification performance to state-of-the-art methodologies. A repeatability experiment using a separate dataset, experimental protocol, and eye tracker was also performed using eight subjects to evaluate the robustness of the proposed approach. The numerical results from both experiments demonstrate that classifiers employing the proposed feature encoding methods outperform the current state-of-the-art in differentiating between radiologists in terms of experience level. This signifies the potential impact of the proposed method for identifying radiologists' level of expertise and those who would benefit from additional training.