Evaluation of Diagnostic Recommendations Embedded in Medication Alerts: Prospective Single-Arm Interventional Study

TL;DR

A machine learning system that suggests diagnoses with medication alerts improved outpatient care by ensuring prescriptions had valid diagnoses.

Contribution

A machine learning–based CDSS with embedded diagnostic recommendations was evaluated for improving prescription appropriateness in outpatient settings.

Findings

The system achieved a 2.28% alert rate and a 56.55% acceptance rate for diagnostic recommendations.

Accepted recommendations led to actionable changes like prescription adjustments or added diagnoses.

Ophthalmology had the highest acceptance rate (96.59%), while some specialties had 0% acceptance.

Abstract

Potentially inappropriate prescribing in outpatient care contributes to adverse outcomes and health care inefficiencies. Clinical decision support systems (CDSS) offer promising solutions, but their effectiveness is often constrained by incomplete medical records.

This study aims to evaluate the effectiveness of a machine learning–based CDSS for enhancing diagnostic recommendations, which are system-suggested diagnoses, ensuring that each prescribed medication has a corresponding diagnosis documented and meets medication appropriateness.

This prospective single-arm interventional study was conducted over 1 year in the outpatient departments of a hospital. The system provided diagnostic recommendations based on machine learning algorithms trained on data from the National Health Insurance Research Database. Outcome measures included alert rates, acceptance rates of diagnostic recommendations, and variability in system performance across specialties. Descriptive and trend analyses were used to evaluate the system’s effectiveness.

This study included 438,558 prescriptions from 44 physicians across 23 specialties, involving 125,000 unique patients in the outpatient departments of a regional teaching hospital. MedGuard, embedded with diagnostic recommendations, achieved an overall alert rate of 2.28% and a diagnostic recommendation acceptance rate of 56.55%. All accepted recommendations resulted in actionable changes, including prescription adjustments or the addition of missing diagnoses. Ophthalmology achieved the highest acceptance rate at 96.59%, while rheumatology, surgery, psychiatry, and infectious disease recorded acceptance rates of 0%, 0%, 24.74%, and 35%, respectively. Over the years, acceptance rates for potentially inappropriate prescriptions stabilized at 51%, despite increasing prescription volumes.

This study demonstrates the potential of embedding diagnostic recommendations into alerts within a machine learning–based clinical decision support system to improve diagnostic completeness and support safer outpatient care. Future efforts should refine alerts to align with specialty-specific workflows and validate their effectiveness in diverse clinical settings.

Figures