Developing and validating a machine learning pharmaceutical therapy recommender system for US-based hospital in-patients with schizophrenia spectrum disorders

TL;DR

This study created a machine learning system to recommend antipsychotic medications for hospitalized schizophrenia patients, showing it works well across different hospitals and time periods.

Contribution

The paper introduces a novel machine learning-based medication recommender system for schizophrenia patients, validated across geographic and temporal contexts.

Findings

Cosine similarity-based collaborative filtering with K=7 neighbors achieved optimal performance in recommending antipsychotic medications.

The model showed high transferability, with second-visit recommendations achieving mean average precision at position three scores up to 0.813.

Clinicians' recommendations outperformed the model in cases of disagreement, leading to better treatment outcomes.

Abstract

No drug recommender system exists to guide antipsychotic medication selection for hospital in-patients with schizophrenia. This study developed and validated a prototype personalised medication recommender system for in-hospital patients with schizophrenia.

This prognostic study analysed data from the Medical Information Mart for Intensive Care IV database on patients prescribed antipsychotic medications with hospital admission diagnoses of schizophrenic disorders between 2008 and 2022. Five similarity-based algorithms (two collaborative filtering and three distance-based representation methods) were evaluated using nested patient-wise cross-validation. The best performing model underwent external validation across geographic and temporal contexts using (a) Northwestern Intensive Care Unit and (b) Medical Information Mart for Intensive Care III datasets. Treatment success was measured through an affinity score (range 0–1) incorporating time between admissions, medication switches, and length of stay. Algorithm performance was assessed using root-mean-square error and mean average precision at position three, with visit-specific analysis across first, second, and third admissions.

Among 1,152 patients (660 first visits, 251 s visits, 241 third visits), cosine similarity-based collaborative filtering with K = 7 neighbours achieved optimal performance. Internal cross-validation demonstrated excellent prediction accuracy (root-mean-square error 0.116, standard deviation 0.007) and moderate recommendation quality (mean average precision at position three 0.490, standard deviation 0.029), improving to 0.697 (standard deviation 0.019) for first visits only. Geographic validation on 32 patients maintained recommendation quality (mean average precision at position three 0.432) despite increased prediction uncertainty (root-mean-square error 0.498). Temporal validation on 66 patients showed similar patterns (mean average precision at position three 0.359, root-mean-square error 0.578). Second-visit recommendations demonstrated exceptional transferability across validation contexts, achieving mean average precision at position three scores up to 0.813. In 163 cases where the model disagreed with attending clinicians, clinicians’ top-3 recommendations achieved significantly superior treatment outcomes (mean difference − 0.122 affinity points, 95% confidence interval: -0.145 to -0.098, p < 0.001, Cohen’s d=-0.81).

This proof-of-concept study demonstrates feasibility of developing a machine learning-based medication recommender system for hospital in-patients with schizophrenia, that appear to maintain recommendation quality across different healthcare institutions and time periods. Prospective clinical trials are needed to establish real-world effectiveness and safety.

The online version contains supplementary material available at 10.1186/s12888-025-07657-8.

Figures