Follow My Eye: Using Gaze to Supervise Computer-Aided Diagnosis

TL;DR

This paper introduces a novel approach that uses radiologists' eye movement data as supervision to train deep neural networks for medical image diagnosis, reducing the need for extensive manual annotations.

Contribution

It presents a new method leveraging expert gaze data to supervise DNN attention, improving diagnostic accuracy in medical imaging tasks.

Findings

Gaze supervision enhances diagnosis performance.

Method achieves notable accuracy improvements.

First to use eye movement as supervision in this context.

Abstract

When deep neural network (DNN) was first introduced to the medical image analysis community, researchers were impressed by its performance. However, it is evident now that a large number of manually labeled data is often a must to train a properly functioning DNN. This demand for supervision data and labels is a major bottleneck in current medical image analysis, since collecting a large number of annotations from experienced experts can be time-consuming and expensive. In this paper, we demonstrate that the eye movement of radiologists reading medical images can be a new form of supervision to train the DNN-based computer-aided diagnosis (CAD) system. Particularly, we record the tracks of the radiologists' gaze when they are reading images. The gaze information is processed and then used to supervise the DNN's attention via an Attention Consistency module. To the best of our knowledge, the above pipeline is among the earliest efforts to leverage expert eye movement for deep-learning-based CAD. We have conducted extensive experiments on knee X-ray images for osteoarthritis assessment. The results show that our method can achieve considerable improvement in diagnosis performance, with the help of gaze supervision.