# Portability of a text mining algorithm for detecting adverse drug reactions in electronic health records across diverse patient groups in two Dutch hospitals

## TL;DR

A text mining algorithm for detecting adverse drug reactions in hospital records works well across different departments and hospitals without needing changes.

## Contribution

The study demonstrates the portability of a text mining algorithm for adverse drug reaction detection across diverse clinical settings.

## Key findings

- The TM algorithm achieved an F-score of 0.64, sensitivity of 80%, and PPV of 54%.
- The algorithm identified 26 clinically relevant adverse drug reactions missed in manual review.
- Performance was consistent across two hospitals and departments without adaptations.

## Abstract

Adverse Drug Reactions (ADRs) pose a significant challenge in healthcare. While structured documentation of ADRs in electronic health records (EHRs) enables automated alerting, many ADRs are recorded as unstructured free-text, limiting detection. Text mining (TM) shows potential for extracting clinically relevant data from unstructured text. However, the portability of TM algorithms across different institutions and departments remains uncertain, due to variations in EHR structures and documentation practices. To enhance these general-purpose algorithms, evaluating their portability is essential for ensuring effective performance across diverse clinical settings. To evaluate the portability of a previously developed TM-based ADR identification algorithm by assessing its performance using EHRs from two different departments in two different hospitals. EHR free-text data from 62 hospitalized patients in the geriatric and orthopedic departments of two Dutch teaching hospitals were reviewed for ADRs via manual review and the TM algorithm. Performance was evaluated using F-score, sensitivity and positive predictive value (PPV), with comparisons across hospitals and departments. Manual review identified 359 unique ADRs. The TM algorithm detected 534 potential ADRs (pADRs), 286 of which overlapped with manual review, yielding an F-score of 0.64, sensitivity of 80% and PPV of 54%. Performance was consistent across hospitals and departments. Notably, 26 pADRs identified by the algorithm were clinically relevant yet missed in manual review. This study demonstrates portability of the TM algorithm by identifying pADRs across different hospitals and departments without adaptations. These findings support its broader implementation potential for ADR detection in diverse healthcare settings.

Adverse Drug Reactions (ADRs) present a significant challenge in healthcare, with many being recorded as unstructured free-text in Electronic Health Records (EHRs). This study evaluates the portability of a text mining (TM) algorithm developed for ADR identification in EHRs, by assessing its performance across two departments in two Dutch hospitals. EHR free-text data from 62 patients in the geriatric and orthopedic departments were analyzed using manual review and the TM algorithm. The results showed that the TM algorithm demonstrated a good performance, with an F-score of 0.64, sensitivity of 80%, and positive predictive value (PPV) of 54%. Additionally, the algorithm identified clinically relevant ADRs that were missed in manual review. These findings suggest that the TM algorithm is portable across different clinical settings without requiring adaptation, highlighting its potential for broader implementation in ADR detection.