Clinical assessment of the criticality index – dynamic, a machine learning prediction model of future care needs in pediatric inpatients

TL;DR

This study analyzed factors linked to accurate and inaccurate ICU care predictions by a machine learning model in pediatric patients to improve its performance.

Contribution

The first comprehensive structured chart review to identify patient and care factors affecting prediction accuracy of the CI-D model.

Findings

False negatives in ICU prediction were older, more often male, and had longer hospital stays.

False positives in ICU discharge prediction were younger and more likely to have respiratory failure as a primary diagnosis.

Demographics and clinical variables did not differ between true negatives and false positives in non-transfer predictions.

Abstract

To assess patient characteristics and care factors that are associated with correct and incorrect predictions of future care locations (ICU vs. non-ICU) by the Criticality Index-Dynamic (CI-D), with the goal of enhancing the CI-D.

Retrospective structured chart review

All pediatric inpatients admitted from January` 1st 2018 – February 29th 2020

through the emergency department.

Patient characteristics and care factors associated with correct (true positives, true negatives) and incorrect predictions (false positives, false negatives) of future care locations (ICU vs. non-ICU) by the CI-D were assessed.

Of the 3,018, patients, 139 transitioned from non-ICU locations to ICU care; 482 were transferred from the ICU to non-ICU care locations, and 2,400 remained in non-ICU care locations. For the ICU Prediction group, the false negative patients were older, more frequently male, and had longer hospital and ICU lengths of stay compared to the true positive patients. The significant differences in the ICU Prediction group for false negative compared to the true positive patients included a less frequent: primary diagnosis of respiratory failure, use of high flow nasal canula, hourly cardio-respiratory vital signs prior to transfer to the ICU, and neurologic vital signs after transfer from the ICU. For the ICU Discharge prediction group, false positive patients were more frequently: younger, had a primary diagnosis of respiratory failure, more frequently received respiratory support after discharge from the ICU, and received less frequent neurological vital signs prior to transfer from the ICU. For the Non-transfer prediction category, demographics and clinical variables did not differ between the true negative and false positive prediction groups.

We conducted the first comprehensive analysis via structured chart reviews of patient characteristics and care factors that are associated with correct and incorrect predictions of future care locations by the machine learning algorithm, the CI-D, gaining insights into potential new predictor variables for inclusion in the model to improve future model iterations.

Figures