Cross Feature Fusion of Fundus Image and Generated Lesion Map for Referable Diabetic Retinopathy Classification

TL;DR

This paper introduces a novel cross-feature fusion method combining fundus images and generated lesion maps using cross-attention, significantly improving diabetic retinopathy classification accuracy for clinical application.

Contribution

It proposes a new cross-learning approach with transfer learning and cross-attention mechanisms, utilizing Swin U-Net for lesion segmentation and enhancing classification performance.

Findings

Achieved 94.6% accuracy on public datasets

Surpassed state-of-the-art methods by 4.4%

Demonstrated effectiveness of cross-attention in feature fusion

Abstract

Diabetic Retinopathy (DR) is a primary cause of blindness, necessitating early detection and diagnosis. This paper focuses on referable DR classification to enhance the applicability of the proposed method in clinical practice. We develop an advanced cross-learning DR classification method leveraging transfer learning and cross-attention mechanisms. The proposed method employs the Swin U-Net architecture to segment lesion maps from DR fundus images. The Swin U-Net segmentation model, enriched with DR lesion insights, is transferred to generate a lesion map. Both the fundus image and its segmented lesion map are used as complementary inputs for the classification model. A cross-attention mechanism is deployed to improve the model's ability to capture fine-grained details from the input pairs. Our experiments, utilizing two public datasets, FGADR and EyePACS, demonstrate a superior accuracy of 94.6%, surpassing current state-of-the-art methods by 4.4%. To this end, we aim for the proposed method to be seamlessly integrated into clinical workflows, enhancing accuracy and efficiency in identifying referable DR.