Improving diabetic retinopathy screening using Artificial Intelligence: design, evaluation and before-and-after study of a custom development

TL;DR

This study developed and evaluated NaIA-RD, a custom AI tool for diabetic retinopathy screening, demonstrating improved sensitivity, reduced workload, and safe autonomous operation in a real-world clinical setting.

Contribution

The paper introduces NaIA-RD, a novel AI system combining DR and image quality grading, and provides a comprehensive before-and-after impact assessment in clinical practice.

Findings

NaIA-RD increased GP screening sensitivity.

High agreement between NaIA-RD and GPs for non-referrals.

NaIA-RD could reduce workload by over 4 times without missing sight-threatening cases.

Abstract

Background: The worst outcomes of diabetic retinopathy (DR) can be prevented by implementing DR screening programs assisted by AI. At the University Hospital of Navarre (HUN), Spain, general practitioners (GPs) grade fundus images in an ongoing DR screening program, referring to a second screening level (ophthalmologist) target patients. Methods: After collecting their requirements, HUN decided to develop a custom AI tool, called NaIA-RD, to assist their GPs in DR screening. This paper introduces NaIA-RD, details its implementation, and highlights its unique combination of DR and retinal image quality grading in a single system. Its impact is measured in an unprecedented before-and-after study that compares 19,828 patients screened before NaIA-RD's implementation and 22,962 patients screened after. Results: NaIA-RD influenced the screening criteria of 3/4 GPs, increasing their sensitivity. Agreement between NaIA-RD and the GPs was high for non-referral proposals (94.6% or more), but lower and variable (from 23.4\% to 86.6%) for referral proposals. An ophthalmologist discarded a NaIA-RD error in most of contradicted referral proposals by labeling the 93% of a sample of them as referable. In an autonomous setup, NaIA-RD would have reduced the study visualization workload by 4.27 times without missing a single case of sight-threatening DR referred by a GP. Conclusion: DR screening was more effective when supported by NaIA-RD, which could be safely used to autonomously perform the first level of screening. This shows how AI devices, when seamlessly integrated into clinical workflows, can help improve clinical pathways in the long term.