GlanceSeg: Real-time microaneurysm lesion segmentation with gaze-map-guided foundation model for early detection of diabetic retinopathy

TL;DR

GlanceSeg is a real-time, gaze-guided segmentation framework for early diabetic retinopathy detection that leverages foundation models and domain knowledge to efficiently identify minute microaneurysm lesions in fundus images.

Contribution

This work introduces GlanceSeg, a novel human-in-the-loop, label-free framework that combines gaze maps and foundation models for precise, real-time microaneurysm segmentation in DR diagnosis.

Findings

GlanceSeg achieves superior segmentation accuracy on public datasets.

The framework improves annotation efficiency for clinicians.

Fine-tuning with annotations enhances segmentation performance.

Abstract

Early-stage diabetic retinopathy (DR) presents challenges in clinical diagnosis due to inconspicuous and minute microangioma lesions, resulting in limited research in this area. Additionally, the potential of emerging foundation models, such as the segment anything model (SAM), in medical scenarios remains rarely explored. In this work, we propose a human-in-the-loop, label-free early DR diagnosis framework called GlanceSeg, based on SAM. GlanceSeg enables real-time segmentation of microangioma lesions as ophthalmologists review fundus images. Our human-in-the-loop framework integrates the ophthalmologist's gaze map, allowing for rough localization of minute lesions in fundus images. Subsequently, a saliency map is generated based on the located region of interest, which provides prompt points to assist the foundation model in efficiently segmenting microangioma lesions. Finally, a domain knowledge filter refines the segmentation of minute lesions. We conducted experiments on two newly-built public datasets, i.e., IDRiD and Retinal-Lesions, and validated the feasibility and superiority of GlanceSeg through visualized illustrations and quantitative measures. Additionally, we demonstrated that GlanceSeg improves annotation efficiency for clinicians and enhances segmentation performance through fine-tuning using annotations. This study highlights the potential of GlanceSeg-based annotations for self-model optimization, leading to enduring performance advancements through continual learning.