Neural Summarization of Electronic Health Records

TL;DR

This study explores neural network models to automatically generate hospital discharge summaries from nursing notes, demonstrating significant improvements in text quality and efficiency through targeted fine-tuning and section-specific training.

Contribution

It introduces a novel approach of section-specific fine-tuning of neural models for discharge summary generation, achieving higher ROUGE scores than previous methods.

Findings

Fine-tuned BART improved ROUGE F1 by 43.6%.

Fine-tuned FLAN-T5 achieved the highest ROUGE score of 45.6.

Summarizing sections separately outperformed entire report summarization.

Abstract

Hospital discharge documentation is among the most essential, yet time-consuming documents written by medical practitioners. The objective of this study was to automatically generate hospital discharge summaries using neural network summarization models. We studied various data preparation and neural network training techniques that generate discharge summaries. Using nursing notes and discharge summaries from the MIMIC-III dataset, we studied the viability of the automatic generation of various sections of a discharge summary using four state-of-the-art neural network summarization models (BART, T5, Longformer and FLAN-T5). Our experiments indicated that training environments including nursing notes as the source, and discrete sections of the discharge summary as the target output (e.g. "History of Present Illness") improve language model efficiency and text quality. According to our findings, the fine-tuned BART model improved its ROUGE F1 score by 43.6% against its standard off-the-shelf version. We also found that fine-tuning the baseline BART model with other setups caused different degrees of improvement (up to 80% relative improvement). We also observed that a fine-tuned T5 generally achieves higher ROUGE F1 scores than other fine-tuned models and a fine-tuned FLAN-T5 achieves the highest ROUGE score overall, i.e., 45.6. For majority of the fine-tuned language models, summarizing discharge summary report sections separately outperformed the summarization the entire report quantitatively. On the other hand, fine-tuning language models that were previously instruction fine-tuned showed better performance in summarizing entire reports. This study concludes that a focused dataset designed for the automatic generation of discharge summaries by a language model can produce coherent Discharge Summary sections.