Quasi-multimodal-based pathophysiological feature learning for retinal disease diagnosis

TL;DR

This paper introduces a unified framework that synthesizes and fuses multimodal retinal imaging data, improving disease classification and grading accuracy while addressing challenges like data heterogeneity and registration complexity.

Contribution

The study presents a novel integrated approach combining data synthesis, modality-specific learning, and adaptive fusion for retinal disease diagnosis, outperforming existing methods.

Findings

Superior multi-label classification performance (F1-score: 0.683, AUC: 0.953)

High accuracy in diabetic retinopathy grading (Accuracy: 0.842, Kappa: 0.861)

Effective data augmentation across medical imaging modalities

Abstract

Retinal diseases spanning a broad spectrum can be effectively identified and diagnosed using complementary signals from multimodal data. However, multimodal diagnosis in ophthalmic practice is typically challenged in terms of data heterogeneity, potential invasiveness, registration complexity, and so on. As such, a unified framework that integrates multimodal data synthesis and fusion is proposed for retinal disease classification and grading. Specifically, the synthesized multimodal data incorporates fundus fluorescein angiography (FFA), multispectral imaging (MSI), and saliency maps that emphasize latent lesions as well as optic disc/cup regions. Parallel models are independently trained to learn modality-specific representations that capture cross-pathophysiological signatures. These features are then adaptively calibrated within and across modalities to perform information pruning and flexible integration according to downstream tasks. The proposed learning system is thoroughly interpreted through visualizations in both image and feature spaces. Extensive experiments on two public datasets demonstrated the superiority of our approach over state-of-the-art ones in the tasks of multi-label classification (F1-score: 0.683, AUC: 0.953) and diabetic retinopathy grading (Accuracy:0.842, Kappa: 0.861). This work not only enhances the accuracy and efficiency of retinal disease screening but also offers a scalable framework for data augmentation across various medical imaging modalities.