Learning to Fuse and Reconstruct Multi-View Graphs for Diabetic Retinopathy Grading

TL;DR

This paper introduces MVGFDR, a novel multi-view graph fusion framework that explicitly models inter-view correlations and disentangles shared and view-specific features for improved diabetic retinopathy grading.

Contribution

The paper proposes a new end-to-end multi-view graph fusion method with a novel graph-based module that captures view correlations and enhances view-invariant feature learning.

Findings

Outperforms existing methods on the MFIDDR dataset

Effectively captures inter-view correlations and view-specific features

Achieves superior accuracy in diabetic retinopathy grading

Abstract

Diabetic retinopathy (DR) is one of the leading causes of vision loss worldwide, making early and accurate DR grading critical for timely intervention. Recent clinical practices leverage multi-view fundus images for DR detection with a wide coverage of the field of view (FOV), motivating deep learning methods to explore the potential of multi-view learning for DR grading. However, existing methods often overlook the inter-view correlations when fusing multi-view fundus images, failing to fully exploit the inherent consistency across views originating from the same patient. In this work, we present MVGFDR, an end-to-end Multi-View Graph Fusion framework for DR grading. Different from existing methods that directly fuse visual features from multiple views, MVGFDR is equipped with a novel Multi-View Graph Fusion (MVGF) module to explicitly disentangle the shared and view-specific visual features. Specifically, MVGF comprises three key components: (1) Multi-view Graph Initialization, which constructs visual graphs via residual-guided connections and employs Discrete Cosine Transform (DCT) coefficients as frequency-domain anchors; (2) Multi-view Graph Fusion, which integrates selective nodes across multi-view graphs based on frequency-domain relevance to capture complementary view-specific information; and (3) Masked Cross-view Reconstruction, which leverages masked reconstruction of shared information across views to facilitate view-invariant representation learning. Extensive experimental results on MFIDDR, by far the largest multi-view fundus image dataset, demonstrate the superiority of our proposed approach over existing state-of-the-art approaches in diabetic retinopathy grading.