OpenClinicalAI: enabling AI to diagnose diseases in real-world clinical settings

TL;DR

This paper introduces OpenClinicalAI, an AI system designed for real-world clinical disease diagnosis, addressing the limitations of existing AI systems that only perform well in controlled, closed settings.

Contribution

The paper presents a novel open, dynamic machine learning framework and a new benchmark, Clinical AIBench, for evaluating AI in real-world clinical scenarios, with a focus on Alzheimer's disease.

Findings

OpenClinicalAI significantly outperforms existing AI systems in real-world settings.

It develops personalized diagnosis strategies to reduce unnecessary tests.

It seamlessly collaborates with clinicians for improved medical services.

Abstract

This paper quantitatively reveals the state-of-the-art and state-of-the-practice AI systems only achieve acceptable performance on the stringent conditions that all categories of subjects are known, which we call closed clinical settings, but fail to work in real-world clinical settings. Compared to the diagnosis task in the closed setting, real-world clinical settings pose severe challenges, and we must treat them differently. We build a clinical AI benchmark named Clinical AIBench to set up real-world clinical settings to facilitate researches. We propose an open, dynamic machine learning framework and develop an AI system named OpenClinicalAI to diagnose diseases in real-world clinical settings. The first versions of Clinical AIBench and OpenClinicalAI target Alzheimer's disease. In the real-world clinical setting, OpenClinicalAI significantly outperforms the state-of-the-art AI system. In addition, OpenClinicalAI develops personalized diagnosis strategies to avoid unnecessary testing and seamlessly collaborates with clinicians. It is promising to be embedded in the current medical systems to improve medical services.