Decentralized LoRA augmented transformer with multi-scale feature learning for secured eye diagnosis

TL;DR

This paper introduces a privacy-preserving, multi-scale transformer framework with LoRA and federated learning for improved ophthalmic disease diagnosis, demonstrating superior accuracy and interpretability on benchmark datasets.

Contribution

It presents a novel integrated framework combining multi-scale patch embedding, LoRA, knowledge distillation, and federated learning for secure and effective eye disease diagnosis.

Findings

Outperforms traditional CNNs and transformer models on OCTDL and Eye Disease Image Dataset.

Achieves higher AUC, F1 score, and precision across evaluations.

Provides interpretable Grad-CAM++ visualizations supporting clinical trust.

Abstract

Accurate and privacy-preserving diagnosis of ophthalmic diseases remains a critical challenge in medical imaging, particularly given the limitations of existing deep learning models in handling data imbalance, data privacy concerns, spatial feature diversity, and clinical interpretability. This paper proposes a novel Data efficient Image Transformer (DeiT) based framework that integrates context aware multiscale patch embedding, Low-Rank Adaptation (LoRA), knowledge distillation, and federated learning to address these challenges in a unified manner. The proposed model effectively captures both local and global retinal features by leveraging multi scale patch representations with local and global attention mechanisms. LoRA integration enhances computational efficiency by reducing the number of trainable parameters, while federated learning ensures secure, decentralized training without compromising data privacy. A knowledge distillation strategy further improves generalization in data scarce settings. Comprehensive evaluations on two benchmark datasets OCTDL and the Eye Disease Image Dataset demonstrate that the proposed framework consistently outperforms both traditional CNNs and state of the art transformer architectures across key metrics including AUC, F1 score, and precision. Furthermore, Grad-CAM++ visualizations provide interpretable insights into model predictions, supporting clinical trust. This work establishes a strong foundation for scalable, secure, and explainable AI applications in ophthalmic diagnostics.