Understanding psychiatric-legal disagreements in not criminally responsible on account of mental disorder cases: a gradient boosting model perspective

TL;DR

This study uses machine learning to explore why mental health tribunals sometimes disagree with psychiatrists' recommendations in criminal cases.

Contribution

A novel application of gradient boosting models to analyze decision-making patterns in psychiatric-legal disagreements.

Findings

The CatBoost model achieved 82% accuracy in predicting psychiatrist–tribunal agreement or disagreement.

SHAP analysis identified prior tribunal decisions and risk factors as key predictors of agreement.

Judicial decisions show a tendency toward path dependence and risk aversion.

Abstract

According to the Canadian Criminal Code, when a court or a mental health review board makes a disposition for an individual found not criminally responsible on account of mental disorder (NCRMD), it must consider several factors: foremost, the safety of the public, as the paramount concern, as well as the mental condition of the accused, their reintegration into society, and their other needs. While psychiatric evaluations are central to these hearings, the CETM does not always follow the psychiatrist’s recommendations. This study aims to identify variables that predict agreement or disagreement between psychiatric recommendations and CETM decisions, using machine learning to better understand this decision-making process.

We retrieved all CETM judgments from 2023 (N = 1,327) from the publicly accessible SOQUIJ database. Cases were included based on NCRMD status and judgment type (initial or annual reviews). A coding framework was developed to extract sociodemographic, clinical, legal, and administrative variables. A CatBoost classification model with SMOTE oversampling was applied to predict psychiatrist–tribunal agreement versus disagreement. Model performance was evaluated using accuracy, precision, recall, F1 score, and AUC. SHAP (SHapley Additive Explanations) values were used to assess variable importance.

The CatBoost model achieved an overall accuracy of 82% and an AUC-ROC of 0.672. The model performed better in identifying agreements (precision: 0.83, recall: 0.98) than disagreements (precision: 0.50, recall: 0.10). SHAP analysis revealed that the most influential predictors of agreement were whether the psychiatrist’s recommendation aligned with the CETM’s previous decision, the presence of high-risk elements, and requests for unconditional release by legal counsel.

Our findings suggest a pattern of judicial path dependence and risk aversion in CETM decisions. Machine learning offers a promising avenue to elucidate decision-making in forensic psychiatric tribunals.

Figures