Progress Notes Classification and Keyword Extraction using Attention-based Deep Learning Models with BERT

TL;DR

This paper explores attention-based deep learning models, particularly fine-tuned BERT, for classifying clinical progress notes and extracting relevant keywords, achieving high accuracy and interpretability despite noisy and unstructured clinical data.

Contribution

It introduces an attention-based BERT model that improves classification accuracy and provides interpretable keyword extraction for clinical notes.

Findings

Achieved 97.6% classification accuracy with fine-tuned BERT and attention.

Attention-based models identify relevant keywords related to note categories.

Models offer interpretability in clinical text classification.

Abstract

Various deep learning algorithms have been developed to analyze different types of clinical data including clinical text classification and extracting information from 'free text' and so on. However, automate the keyword extraction from the clinical notes is still challenging. The challenges include dealing with noisy clinical notes which contain various abbreviations, possible typos, and unstructured sentences. The objective of this research is to investigate the attention-based deep learning models to classify the de-identified clinical progress notes extracted from a real-world EHR system. The attention-based deep learning models can be used to interpret the models and understand the critical words that drive the correct or incorrect classification of the clinical progress notes. The attention-based models in this research are capable of presenting the human interpretable text classification models. The results show that the fine-tuned BERT with the attention layer can achieve a high classification accuracy of 97.6%, which is higher than the baseline fine-tuned BERT classification model. In this research, we also demonstrate that the attention-based models can identify relevant keywords that are strongly related to the clinical progress note categories.