# Utilizing large language models and natural language processing to classify ischemia status from cardiac stress tests in a large multicenter healthcare system

## TL;DR

This study uses natural language processing to accurately classify cardiac stress test reports for signs of ischemia, enabling efficient quality-of-care assessments.

## Contribution

A rules-based NLP system and ClinicalBERT model were developed and validated for ischemia classification in stress test reports.

## Key findings

- ClinicalBERT achieved 86.4% precision, 100% recall, and 92.7% F1 score in ischemia classification.
- The rules-based system matched high performance with 88.1% precision, 97.4% recall, and 92.5% F1 score.
- Over 1.6 million stress test reports were classified using the rules-based system for quality evaluations.

## Abstract

Documentation of myocardial ischemia prior to invasive coronary angiography is recommended to minimize patient risk. However, obtaining this information for quality-of-care assessment often requires extracting clinical information from unstructured electronic medical records text. To this end, we sought to evaluate multiple natural language processing (NLP) systems in their ability to classify cardiac stress test reports as documenting ischemia or no ischemia, implementing the one with the best combination of accuracy and feasibility.

Four BERT large language models (LLMs) were fine-tuned, and a rules-based system was designed by training, validating, and testing on an annotated sample of 654 stress test reports from a multisite and multiyear dataset from the Veterans Health Administration (VHA). The LLM with the highest performance was a ClinicalBERT with precision, recall, and F1 of 86.4%, 100%, and 92.7%, respectively. The rules-based NLP system achieved similar results of 88.1%, 97.4%, and 92.5%, respectively. Stress test reports totaling 1,692,171 and representing 1,096,341 unique patients were classified using the rules-based system after ascertaining current technological limitations, and the system is presently operational for care quality evaluations. Utilizing NLP allows for accurate, high-throughput analysis of cardiac stress test text reports.

The online version contains supplementary material available at 10.1186/s13104-025-07586-5.