Localization supervision of chest x-ray classifiers using label-specific eye-tracking annotation

TL;DR

This paper introduces a novel method using eye-tracking data from radiologists to supervise the localization of abnormalities in chest X-ray classifiers, enhancing interpretability without affecting classification performance.

Contribution

It demonstrates that eye-tracking data can effectively supervise CNN localization in chest X-rays, reducing the need for costly bounding box annotations.

Findings

Improved interpretability of CNNs with eye-tracking supervision

No negative impact on classification accuracy

Effective localization of abnormalities using eye-tracking data

Abstract

Convolutional neural networks (CNNs) have been successfully applied to chest x-ray (CXR) images. Moreover, annotated bounding boxes have been shown to improve the interpretability of a CNN in terms of localizing abnormalities. However, only a few relatively small CXR datasets containing bounding boxes are available, and collecting them is very costly. Opportunely, eye-tracking (ET) data can be collected in a non-intrusive way during the clinical workflow of a radiologist. We use ET data recorded from radiologists while dictating CXR reports to train CNNs. We extract snippets from the ET data by associating them with the dictation of keywords and use them to supervise the localization of specific abnormalities. We show that this method improves a model's interpretability without impacting its image-level classification.