Comparison of pharmacist evaluation of medication orders with predictions of a machine learning model

TL;DR

This study evaluates an unsupervised machine learning model's ability to identify unusual medication orders and profiles, comparing its predictions with clinical pharmacists' assessments in a real-world setting.

Contribution

It provides a prospective evaluation of a machine learning model's clinical performance in detecting atypical medication orders and profiles.

Findings

Poor performance in identifying unusual medication orders based on AUPR.

Satisfactory performance in identifying atypical pharmacological profiles.

Pharmacists found the model useful as a screening tool.

Abstract

The objective of this work was to assess the clinical performance of an unsupervised machine learning model aimed at identifying unusual medication orders and pharmacological profiles. We conducted a prospective study between April 2020 and August 2020 where 25 clinical pharmacists dichotomously (typical or atypical) rated 12,471 medication orders and 1,356 pharmacological profiles. Based on AUPR, performance was poor for orders, but satisfactory for profiles. Pharmacists considered the model a useful screening tool.