Diabetic Retinopathy Grading with CLIP-based Ranking-Aware Adaptation:A Comparative Study on Fundus Image

TL;DR

This study compares three CLIP-based methods for diabetic retinopathy grading, demonstrating that ranking-aware prompting and hybrid models significantly outperform zero-shot approaches in accuracy and clinical relevance.

Contribution

It introduces and evaluates three CLIP-based approaches, including a ranking-aware model, for multi-class DR severity grading on fundus images, highlighting their effectiveness.

Findings

Ranking-aware model achieves 93.42% accuracy and AUROC 0.9845.

Hybrid FCN-CLIP model achieves 92.49% accuracy and AUROC 0.99.

Both models outperform the zero-shot baseline by a large margin.

Abstract

Diabetic retinopathy (DR) is a leading cause of preventable blindness, and automated fundus image grading can play an important role in large-scale screening. In this work, we investigate three CLIP-based approaches for five-class DR severity grading: (1) a zero-shot baseline using prompt engineering, (2) a hybrid FCN-CLIP model augmented with CBAM attention, and (3) a ranking-aware prompting model that encodes the ordinal structure of DR progression. We train and evaluate on a combined dataset of APTOS 2019 and Messidor-2 (n=5,406), addressing class imbalance through resampling and class-specific optimal thresholding. Our experiments show that the ranking-aware model achieves the highest overall accuracy (93.42%, AUROC 0.9845) and strong recall on clinically critical severe cases, while the hybrid FCN-CLIP model (92.49%, AUROC 0.99) excels at detecting proliferative DR. Both substantially outperform the zero-shot baseline (55.17%, AUROC 0.75). We analyze the complementary strengths of each approach and discuss their practical implications for screening contexts.