Transforming unstructured voice and text data into insight for paramedic emergency service using recurrent and convolutional neural networks

TL;DR

This paper presents a system that automatically fuses voice and text data using neural networks to provide real-time medical insights and incident reports, enhancing paramedic decision-making during emergencies.

Contribution

It introduces a novel deep learning approach combining LSTM and CNN models to classify and generate incident reports from unstructured voice and text data in emergency scenarios.

Findings

Timely medication notifications achieved from unstructured data

Automatic incident report generation improves decision-making

System reduces error-prone manual tasks for paramedics and doctors

Abstract

Paramedics often have to make lifesaving decisions within a limited time in an ambulance. They sometimes ask the doctor for additional medical instructions, during which valuable time passes for the patient. This study aims to automatically fuse voice and text data to provide tailored situational awareness information to paramedics. To train and test speech recognition models, we built a bidirectional deep recurrent neural network (long short-term memory (LSTM)). Then we used convolutional neural networks on top of custom-trained word vectors for sentence-level classification tasks. Each sentence is automatically categorized into four classes, including patient status, medical history, treatment plan, and medication reminder. Subsequently, incident reports were automatically generated to extract keywords and assist paramedics and physicians in making decisions. The proposed system found that it could provide timely medication notifications based on unstructured voice and text data, which was not possible in paramedic emergencies at present. In addition, the automatic incident report generation provided by the proposed system improves the routine but error-prone tasks of paramedics and doctors, helping them focus on patient care.