Machine Learning for Violence Risk Assessment Using Dutch Clinical Notes

TL;DR

This study evaluates machine learning models, including deep learning and conventional methods, for violence risk assessment using Dutch clinical notes, achieving comparable performance to existing questionnaire-based tools.

Contribution

It compares the effectiveness of deep learning and traditional machine learning on clinical notes for violence risk prediction, highlighting the limitations of deep models like BERTje.

Findings

Best models achieved AUC of ~0.8

Deep BERTje performed worse than conventional methods

Data and classifier evaluation aids applicability to new data

Abstract

Violence risk assessment in psychiatric institutions enables interventions to avoid violence incidents. Clinical notes written by practitioners and available in electronic health records are valuable resources capturing unique information, but are seldom used to their full potential. We explore conventional and deep machine learning methods to assess violence risk in psychiatric patients using practitioner notes. The performance of our best models is comparable to the currently used questionnaire-based method, with an area under the Receiver Operating Characteristic curve of approximately 0.8. We find that the deep-learning model BERTje performs worse than conventional machine learning methods. We also evaluate our data and our classifiers to understand the performance of our models better. This is particularly important for the applicability of evaluated classifiers to new data, and is also of great interest to practitioners, due to the increased availability of new data in electronic format.