Advancing Psychiatric Safety With the Predictive Risk Identification for Mental Health Events Tool: Retrospective Cohort Study

TL;DR

A new deep learning tool called PRIME improves prediction of mental health events in psychiatric hospitals, outperforming existing methods.

Contribution

PRIME is one of the first deep learning-based early warning systems for psychiatric inpatient care, offering interpretable and proactive risk predictions.

Findings

PRIME achieved an AUC of 0.83, significantly outperforming DASA-IV by 0.20 in predicting adverse mental health events.

The model demonstrated potential for proactive risk management through interpretable, rolling predictions.

Long short-term memory with attention mechanisms provided the highest predictive performance in the study.

Abstract

Patient safety incidents are a leading cause of harm in psychiatric settings, yet early warning systems (EWS) tailored to mental health remain underdeveloped. Traditional risk tools such as the Dynamic Appraisal of Situational Aggression–Inpatient Version (DASA-IV) offer limited predictive accuracy and are reactive rather than proactive.

We introduce the Predictive Risk Identification for Mental Health Events (PRIME) tool, a deep learning–based EWS trained on longitudinal psychiatric electronic medical record (EMR) data to anticipate adverse events in 24-hour windows.

A retrospective cohort study using routinely collected EMR data to train and validate machine learning (ML) models for short-term risk prediction was conducted. This study took place at Waypoint Centre for Mental Health Care, a large inpatient psychiatric hospital in Ontario, Canada, serving both high-security forensic and nonforensic patient populations. A total of 4651 patients and 403,098 encounters from January 2020 to August 2024 were included. For model evaluation, the 2024 test set included 900 patients and 48,313 encounters. PRIME was trained using recurrent neural networks with attention mechanisms on multivariate time-series data. The model used an autoregressive design to forecast risk based on 7 days of prior patient data and was benchmarked against the DASA-IV clinical tool and other ML baselines. The primary outcome was the occurrence of an adverse mental health event recorded in the EMR within the following 24 hours. Model performance was assessed using area under the receiver operating characteristic curve (AUC) and recall, alongside subgroup analyses and interpretability assessments using integrated gradients.

The long short-term memory with attention mechanism achieved the highest predictive performance (AUC=0.83), outperforming existing tools such as DASA-IV by 0.20 AUC (0.81 vs 0.61) and demonstrating the potential of ML-based models to support proactive risk management in mental health settings.

The PRIME tool is one of the first developed and evaluated deep learning–based EWS for psychiatric inpatient care. By outperforming existing clinical tools and providing interpretable, rolling predictions, PRIME offers a pathway toward safer, more proactive mental health interventions. Future work should assess its equity implications and integration into routine psychiatric workflows.

Figures