A Machine Learning Approach for Emergency Detection in Medical Scenarios Using Large Language Models

TL;DR

This paper introduces a novel system using large language models and prompt engineering to accurately and rapidly detect medical emergencies in digital communications, enhancing telemedicine safety.

Contribution

It demonstrates the effective use of LLMs with prompt engineering for emergency detection, achieving near-perfect accuracy and providing a publicly available evaluation framework.

Findings

LLaMA 2 (7B) achieved 99.7% accuracy

Optimal performance with 10 prompt examples

Processing speed varies from 0.05 to 2.2 seconds

Abstract

The rapid identification of medical emergencies through digital communication channels remains a critical challenge in modern healthcare delivery, particularly with the increasing prevalence of telemedicine. This paper presents a novel approach leveraging large language models (LLMs) and prompt engineering techniques for automated emergency detection in medical communications. We developed and evaluated a comprehensive system using multiple LLaMA model variants (1B, 3B, and 7B parameters) to classify medical scenarios as emergency or non-emergency situations. Our methodology incorporated both system prompts and in-prompt training approaches, evaluated across different hardware configurations. The results demonstrate exceptional performance, with the LLaMA 2 (7B) model achieving 99.7% accuracy and the LLaMA 3.2 (3B) model reaching 99.6% accuracy with optimal prompt engineering. Through systematic testing of training examples within the prompts, we identified that including 10 example scenarios in the model prompts yielded optimal classification performance. Processing speeds varied significantly between platforms, ranging from 0.05 to 2.2 seconds per request. The system showed particular strength in minimizing high-risk false negatives in emergency scenarios, which is crucial for patient safety. The code implementation and evaluation framework are publicly available on GitHub, facilitating further research and development in this crucial area of healthcare technology.