# Comparing the effectiveness of a medication knowledge base product as designed with real-world hospital implementations using the Leapfrog Group’s Computerized Physician Order Entry (CPOE/EHR) Evaluation Tool

## TL;DR

This study compares how well a medication knowledge base product works when used as designed versus in real-world hospital settings.

## Contribution

The study reveals that hospital-level configuration choices have more impact on performance than the EHR vendor used.

## Key findings

- The fully enabled MKB product achieved a 94% score, while hospitals averaged 61.1% to 68.8%.
- EHR Vendor B scored 3.6 percentage points lower than Vendor A, but this explained only 3.7% of score variation.
- Implementation and configuration choices at the hospital level significantly affect performance.

## Abstract

We sought to compare and analyze the performance of a medication knowledge base (MKB) vendor’s (Medi-Span) fully enabled, factory settings product to operational implementations in hospitals using the Leapfrog Group’s Computerized Physician Order Entry (CPOE)/electronic health record (EHR) Evaluation Tool.

We randomly selected a hospital that used the MKB vendor’s product to complete the Leapfrog CPOE/EHR Evaluation Tool in 2018 and used this version of the tool to assess the MKB vendor’s fully enabled product. Next, we retrospectively evaluated all hospitals, regardless of EHR vendor, that used the MKB vendor’s product in the tool from 2017 to 2019. We fit a multivariate linear regression model with hospitals’ overall scores as the dependent variable and EHR vendor as the independent variable.

The fully enabled version of the MKB vendor achieved a 94% overall score in the tool. The mean overall score across 358 hospitals from 2017 to 2019 ranged from 61.1% to 68.8%. Finally, the regression model revealed EHR Vendor B hospitals scored 3.6 percentage points lower (P = .01) than EHR Vendor A hospitals, but the model only explained 3.7% of the variation in overall scores.

The study revealed variations in the effectiveness of implementation and configuration of the MKB vendor’s product across all EHR vendors at the hospital level.

Our regression model found that EHR vendor had minimal effect on the overall tool scores. Therefore, EHR vendor is not nearly as important in the performance on the CPOE/EHR Evaluation Tool as the choices made in configuring the MKB vendor’s product within a specific EHR.