EyeDiff: text-to-image diffusion model improves rare eye disease diagnosis

TL;DR

EyeDiff is a novel text-to-image diffusion model that generates multimodal ophthalmic images from natural language prompts, improving diagnosis accuracy for common and rare eye diseases by addressing data scarcity and imbalance.

Contribution

The paper introduces EyeDiff, a new diffusion-based model that creates realistic ophthalmic images from text, aiding rare disease diagnosis and overcoming data collection challenges.

Findings

Generated images accurately reflect lesion features.

Enhanced detection accuracy for rare and minority eye diseases.

Outperforms traditional oversampling in data imbalance scenarios.

Abstract

The rising prevalence of vision-threatening retinal diseases poses a significant burden on the global healthcare systems. Deep learning (DL) offers a promising solution for automatic disease screening but demands substantial data. Collecting and labeling large volumes of ophthalmic images across various modalities encounters several real-world challenges, especially for rare diseases. Here, we introduce EyeDiff, a text-to-image model designed to generate multimodal ophthalmic images from natural language prompts and evaluate its applicability in diagnosing common and rare diseases. EyeDiff is trained on eight large-scale datasets using the advanced latent diffusion model, covering 14 ophthalmic image modalities and over 80 ocular diseases, and is adapted to ten multi-country external datasets. The generated images accurately capture essential lesional characteristics, achieving high alignment with text prompts as evaluated by objective metrics and human experts. Furthermore, integrating generated images significantly enhances the accuracy of detecting minority classes and rare eye diseases, surpassing traditional oversampling methods in addressing data imbalance. EyeDiff effectively tackles the issue of data imbalance and insufficiency typically encountered in rare diseases and addresses the challenges of collecting large-scale annotated images, offering a transformative solution to enhance the development of expert-level diseases diagnosis models in ophthalmic field.