REFLACX, a dataset of reports and eye-tracking data for localization of abnormalities in chest x-rays

TL;DR

The paper introduces REFLACX, a scalable dataset combining eye-tracking and report data for improved localization of abnormalities in chest X-rays, aiding deep learning models.

Contribution

It presents a novel method for collecting implicit localization data via eye tracking and speech, creating a large, annotated dataset for chest X-ray analysis.

Findings

REFLACX contains 3,032 eye-tracking and report data sets for 2,616 X-rays.

Provides auxiliary annotations including bounding boxes and abnormality localizations.

Includes inter-rater scores from multiple radiologists.

Abstract

Deep learning has shown recent success in classifying anomalies in chest x-rays, but datasets are still small compared to natural image datasets. Supervision of abnormality localization has been shown to improve trained models, partially compensating for dataset sizes. However, explicitly labeling these anomalies requires an expert and is very time-consuming. We propose a potentially scalable method for collecting implicit localization data using an eye tracker to capture gaze locations and a microphone to capture a dictation of a report, imitating the setup of a reading room. The resulting REFLACX (Reports and Eye-Tracking Data for Localization of Abnormalities in Chest X-rays) dataset was labeled across five radiologists and contains 3,032 synchronized sets of eye-tracking data and timestamped report transcriptions for 2,616 chest x-rays from the MIMIC-CXR dataset. We also provide auxiliary annotations, including bounding boxes around lungs and heart and validation labels consisting of ellipses localizing abnormalities and image-level labels. Furthermore, a small subset of the data contains readings from all radiologists, allowing for the calculation of inter-rater scores.