Eye tracking guided deep multiple instance learning with dual cross-attention for fundus disease detection

TL;DR

This paper introduces a novel human-in-the-loop CAD system for fundus disease detection that leverages ophthalmologists' eye-tracking data within a dual-cross-attention multiple instance learning framework, improving accuracy and robustness.

Contribution

It proposes a new eye-tracking guided MIL approach with dual-cross-attention and data augmentation modules, integrating medical prior knowledge into CAD systems for fundus diseases.

Findings

Demonstrates the effectiveness of eye-tracking data in improving diagnosis accuracy.

Shows the superiority of the DCAMIL network over baseline models.

Validates the approach on newly constructed AMD-Gaze and DR-Gaze datasets.

Abstract

Deep neural networks (DNNs) have promoted the development of computer aided diagnosis (CAD) systems for fundus diseases, helping ophthalmologists reduce missed diagnosis and misdiagnosis rate. However, the majority of CAD systems are data-driven but lack of medical prior knowledge which can be performance-friendly. In this regard, we innovatively proposed a human-in-the-loop (HITL) CAD system by leveraging ophthalmologists' eye-tracking information, which is more efficient and accurate. Concretely, the HITL CAD system was implemented on the multiple instance learning (MIL), where eye-tracking gaze maps were beneficial to cherry-pick diagnosis-related instances. Furthermore, the dual-cross-attention MIL (DCAMIL) network was utilized to curb the adverse effects of noisy instances. Meanwhile, both sequence augmentation module and domain adversarial module were introduced to enrich and standardize instances in the training bag, respectively, thereby enhancing the robustness of our method. We conduct comparative experiments on our newly constructed datasets (namely, AMD-Gaze and DR-Gaze), respectively for the AMD and early DR detection. Rigorous experiments demonstrate the feasibility of our HITL CAD system and the superiority of the proposed DCAMIL, fully exploring the ophthalmologists' eye-tracking information. These investigations indicate that physicians' gaze maps, as medical prior knowledge, is potential to contribute to the CAD systems of clinical diseases.