Enhancing Diagnostic Reliability of Foundation Model with Uncertainty Estimation in OCT Images

TL;DR

This paper introduces a foundation model with uncertainty estimation for OCT images that improves diagnostic accuracy, detects unseen classes, and outperforms existing algorithms and ophthalmologists, enhancing clinical reliability.

Contribution

The study develops FMUE, a novel foundation model with uncertainty estimation for retinal disease detection in OCT images, demonstrating superior performance and reliability in real-world settings.

Findings

FMUE achieved 98.44% F1 score with thresholding.

FMUE outperformed state-of-the-art algorithms and ophthalmologists.

Model correctly predicts high uncertainty for ambiguous or low-quality samples.

Abstract

Inability to express the confidence level and detect unseen classes has limited the clinical implementation of artificial intelligence in the real-world. We developed a foundation model with uncertainty estimation (FMUE) to detect 11 retinal conditions on optical coherence tomography (OCT). In the internal test set, FMUE achieved a higher F1 score of 96.76% than two state-of-the-art algorithms, RETFound and UIOS, and got further improvement with thresholding strategy to 98.44%. In the external test sets obtained from other OCT devices, FMUE achieved an accuracy of 88.75% and 92.73% before and after thresholding. Our model is superior to two ophthalmologists with a higher F1 score (95.17% vs. 61.93% &71.72%). Besides, our model correctly predicts high uncertainty scores for samples with ambiguous features, of non-target-category diseases, or with low-quality to prompt manual checks and prevent misdiagnosis. FMUE provides a trustworthy method for automatic retinal anomalies detection in the real-world clinical open set environment.