Balancing Natural Language Processing Accuracy and Normalisation in Extracting Medical Insights

TL;DR

This paper compares rule-based NLP methods and Large Language Models for extracting medical insights from Polish clinical texts, highlighting their respective strengths, limitations, and the impact of translation and normalization on accuracy.

Contribution

It provides a comparative analysis of rule-based and LLM approaches for clinical information extraction in low-resource, non-English settings, proposing hybrid solutions.

Findings

Rule-based methods achieve higher accuracy in demographic extraction.

LLMs demonstrate greater adaptability and excel in drug name recognition.

Translation impacts LLM performance and information retention.

Abstract

Extracting structured medical insights from unstructured clinical text using Natural Language Processing (NLP) remains an open challenge in healthcare, particularly in non-English contexts where resources are scarce. This study presents a comparative analysis of NLP low-compute rule-based methods and Large Language Models (LLMs) for information extraction from electronic health records (EHR) obtained from the Voivodeship Rehabilitation Hospital for Children in Ameryka, Poland. We evaluate both approaches by extracting patient demographics, clinical findings, and prescribed medications while examining the effects of lack of text normalisation and translation-induced information loss. Results demonstrate that rule-based methods provide higher accuracy in information retrieval tasks, particularly for age and sex extraction. However, LLMs offer greater adaptability and scalability, excelling in drug name recognition. The effectiveness of the LLMs was compared with texts originally in Polish and those translated into English, assessing the impact of translation. These findings highlight the trade-offs between accuracy, normalisation, and computational cost when deploying NLP in healthcare settings. We argue for hybrid approaches that combine the precision of rule-based systems with the adaptability of LLMs, offering a practical path toward more reliable and resource-efficient clinical NLP in real-world hospitals.