Chief complaint classification with recurrent neural networks

TL;DR

This study compares recurrent neural network models with traditional classifiers for classifying emergency department records, demonstrating that RNNs outperform bag-of-words methods in syndromic surveillance tasks.

Contribution

It introduces LSTM and GRU-based RNN models for syndrome classification and shows their superior performance over traditional classifiers on large-scale emergency data.

Findings

RNN models outperform bag-of-words classifiers in F1 score.

LSTM performs best with discharge diagnoses; GRU excels with chief complaints.

Certain syndrome types are more accurately predicted than others.

Abstract

Syndromic surveillance detects and monitors individual and population health indicators through sources such as emergency department records. Automated classification of these records can improve outbreak detection speed and diagnosis accuracy. Current syndromic systems rely on hand-coded keyword-based methods to parse written fields and may benefit from the use of modern supervised-learning classifier models. In this paper we implement two recurrent neural network models based on long short-term memory (LSTM) and gated recurrent unit (GRU) cells and compare them to two traditional bag-of-words classifiers: multinomial naive Bayes (MNB) and a support vector machine (SVM). The MNB classifier is one of only two machine learning algorithms currently being used for syndromic surveillance. All four models are trained to predict diagnostic code groups as defined by Clinical Classification Software, first to predict from discharge diagnosis, then from chief complaint fields. The classifiers are trained on 3.6 million de-identified emergency department records from a single United States jurisdiction. We compare performance of these models primarily using the F1 score. Using discharge diagnoses, the LSTM classifier performs best, though all models exhibit an F1 score above 96.00. The GRU performs best on chief complaints (F1=47.38), and MNB with bigrams performs worst (F1=39.40). Certain syndrome types are easier to detect than others. For examples, chief complaints using the GRU model predicts alcohol-related disorders well (F1=78.91) but predicts influenza poorly (F1=14.80). In all instances, the RNN models outperformed the bag-of-word classifiers, suggesting deep learning models could substantially improve the automatic classification of unstructured text for syndromic surveillance.