Prediction cardiovascular deterioration in a paediatric intensive care unit (PicEWS): a machine learning modelling study of routinely collected health-care data

TL;DR

This study uses machine learning to predict cardiovascular deterioration in critically ill children, outperforming existing clinical methods.

Contribution

A novel machine learning model (PicEWS) that uses age-normalized and time-variant features to predict cardiovascular deterioration in PICU patients.

Findings

PicEWS predicted cardiovascular deterioration in 90% of cases with fewer than two false alarms per true alarm.

XGBoost outperformed other models and clinical scores like pSOFA in prediction accuracy.

Key features included blood pressure, bilirubin, and COMFORT score, with variability over time being crucial for model performance.

Abstract

Paediatric intensive care medicine uses fine granular clinical data that describe substantial patient instability to make high-consequence decisions. However, these decisions are also hindered by clinical experts’ ability to interpret longitudinal data along with recent and gradual changes in the vital sign data. Machine learning aided decisions can improve the identification of patient deterioration. Important prior work has predicted outcomes in paediatric intensive care units (PICUs), but has often used non-time series data without age normalisation. Most current work also aims to predict mortality, not potentially treatable clinical inflection points such as cardiovascular deterioration.

We extracted telemetry data, alongside laboratory and demographic data, from the Electronic Health Record (EHR) of patients admitted to the general PICU at Great Ormond Street Hospital, London (UK), between 1st April 2019 and 31st April 2021. We engineered deterioration monitoring variables into a smaller feature set using a generalisable pipeline. We calculated trend and variability, and used validated age-normalisation for input variables where appropriate. We compared neural network models, gradient-boosted decision trees (XGBoost), and a logistic regression model to predict cardiovascular deterioration within 12 h (defined as a rise in the paediatric Sequential Organ Failure Assessment (pSOFA) cardiovascular sub-score, rising plasma lactate if lactate ≥2 mmol/l, new extra-corporeal membrane oxygenation (ECMO) requirement, or death). We trained the models on a 70-15-15 percent train-test-validation split. We compared model compositions, including without trend, variability, and frequency of input to smaller models. We investigated feature importance using internal feature importance and Shapley Additive Explanation values. We compared the resulting paediatric intensive care early warning score (PicEWS) with the paediatric Sequential Organ Failure Assessment (pSOFA) score as the gold-standard.

1167 patients were included out of a possible 1195. The best performing predictive model for PicEWS was XGBoost. PicEWS was able to predict cardiovascular deterioration 90% of the time, with fewer than two false alarms for every true alarm. For this model, the area under the precision-recall curve (AUPRC) was 0.552, and area under the receiver operator curve (AUROC) was 0.949. This outperformed pSOFA, which yielded over 10 false alarms per true alarm, with an AUPRC of 0.150 and AUROC of 0.715. The most important features for PicEWS included blood pressure, physiological markers such as bilirubin, and COMFORT score (a sedation and behavioural score used in paediatric intensive care). Feature variability was key to model performance. We demonstrated predictions on an individual patient to show model utility. The study showed that machine learning models can outperform current clinical best practice approaches. We use our model to provide insights into future improvements in clinical practice.

PicEWS outperforms current clinical modelling approaches to predict cardiovascular deterioration. The proposed data processing pipeline and machine learning method offer a clinically applicable decision-support model using age-stratified normal ranges and feature variability over time for the early detection of clinical deterioration in critically ill children.

The NIHR Great Ormond Street Biomedical Research Centre at UCL and the 10.13039/501100001279Great Ormond Street Hospital Children's Charity peer-reviewed grant award.

Figures