# Inter-clinician diagnostic agreement of shock etiology: a multicenter observational study

## TL;DR

This study found that clinicians often disagree on the cause of shock in ICU patients, and machine learning can predict these disagreements.

## Contribution

The study introduces a novel method to quantify diagnostic disagreement using cosine similarity and machine learning prediction models.

## Key findings

- 18.2% of patients had no complete inter-clinician diagnostic agreement for shock etiology.
- Machine learning models achieved high accuracy in predicting patients with diagnostic disagreement.
- Patients without agreement had higher mortality and fewer comorbidities.

## Abstract

We sought to (1) quantify lack of inter-clinician diagnostic agreement of shock etiology and (2) predict patients without complete inter-clinician diagnostic agreement of shock etiology.

This multicenter retrospective, cohort study identified patients evaluated by two or more clinicians who documented a shock diagnosis from 2018 to 2023 across intensive care units (ICU) at 9 acute care hospitals. Shock etiology was abstracted using regular expression from clinician notes in the electronic health record then was made into a 9-dimensional vector representing 9 different shock etiologies. Inter-clinician diagnostic agreement of these vectors was calculated for each patient using Cosine Similarity Scores. Measure of agreement was based on cosine similarity of etiology vectors, not clinical adjudication. Patients without complete inter-clinician diagnostic agreement (Cosine Similarity Score < 1) were compared to patients with diagnostic agreement. Machine learning models were tested to predict patients without complete inter-clinician diagnostic agreement.

Of 7302 patients, 1327 (18.2%) never had complete inter-clinician diagnostic agreement. Patients without diagnostic agreement were more often Black (20.5 vs 16.2%, p = 0.011), with more comorbidities (Elixhauser Comorbidity Index > 10; 39.1 vs 31.6%, p < 0.001), and Sequential Organ Failure Assessment (SOFA) score > 15 (12.1 vs 7.6%, p < 0.001). Patients without diagnostic agreement less frequently had improvements in SOFA scores between ICU days 0 and 4 (34.7 vs 41.9%, p < 0.001), and more often died in-hospital (41.5 vs. 27.6%, p < 0.001). Machine learning models that most accurately predicted patients without diagnostic agreement were logistic regression (Accuracy: 0.8597, F1-Score: 0.9117, AUC-ROC: 0.9257), random forest (Accuracy: 0.8658, F1-Score: 0.9201, AUC-ROC: 0.9255), and gradient boosting (Accuracy: 0.8515, F1-Score: 0.9138, AUC-ROC: 0.9227).

Patients without complete inter-clinician diagnostic agreement of shock etiology can be successfully predicted.

The online version contains supplementary material available at 10.1007/s13755-025-00423-w.