Temporal-Enhanced Interpretable Multi-Modal Prognosis and Risk Stratification Framework for Diabetic Retinopathy (TIMM-ProRS)

TL;DR

The paper presents TIMM-ProRS, a deep learning framework that combines multiple neural network architectures and multi-modal data to improve early diagnosis and risk stratification of diabetic retinopathy, achieving high accuracy and interpretability.

Contribution

Introduces a novel multi-modal, multi-architecture deep learning framework for diabetic retinopathy prognosis that integrates retinal images and temporal biomarkers for enhanced diagnosis.

Findings

Achieves 97.8% accuracy and 0.96 F1-score on diverse datasets.

Outperforms existing methods like RSG-Net and DeepDR.

Enables early, interpretable diagnosis for scalable telemedical use.

Abstract

Diabetic retinopathy (DR), affecting millions globally with projections indicating a significant rise, poses a severe blindness risk and strains healthcare systems. Diagnostic complexity arises from visual symptom overlap with conditions like age-related macular degeneration and hypertensive retinopathy, exacerbated by high misdiagnosis rates in underserved regions. This study introduces TIMM-ProRS, a novel deep learning framework integrating Vision Transformer (ViT), Convolutional Neural Network (CNN), and Graph Neural Network (GNN) with multi-modal fusion. TIMM-ProRS uniquely leverages both retinal images and temporal biomarkers (HbA1c, retinal thickness) to capture multi-modal and temporal dynamics. Evaluated comprehensively across diverse datasets including APTOS 2019 (trained), Messidor-2, RFMiD, EyePACS, and Messidor-1 (validated), the model achieves 97.8\% accuracy and an F1-score of 0.96, demonstrating state-of-the-art performance and outperforming existing methods like RSG-Net and DeepDR. This approach enables early, precise, and interpretable diagnosis, supporting scalable telemedical management and enhancing global eye health sustainability.