Automated Auditing of Hospital Discharge Summaries for Care Transitions

TL;DR

This paper introduces an automated framework leveraging locally deployed Large Language Models to efficiently audit hospital discharge summaries, aiming to improve documentation quality and patient safety at scale.

Contribution

It presents a novel, scalable approach using privacy-preserving LLMs to systematically evaluate discharge summaries against a structured checklist.

Findings

Demonstrated feasibility of automated auditing on MIMIC-IV data.

Achieved accurate identification of key documentation elements.

Laid groundwork for systematic quality improvement in EHR documentation.

Abstract

Incomplete or inconsistent discharge documentation is a primary driver of care fragmentation and avoidable readmissions. Despite its critical role in patient safety, auditing discharge summaries relies heavily on manual review and is difficult to scale. We propose an automated framework for large-scale auditing of discharge summaries using locally deployed Large Language Models (LLMs). Our approach operationalizes core transition-of-care requirements such as follow-up instructions, medication history and changes, patient information and clinical course, etc. into a structured validation checklist of questions based on DISCHARGED framework. Using adult inpatient summaries from the MIMIC-IV database, we utilize a privacy-preserving LLM to identify the presence, absence, or ambiguity of key documentation elements. This work demonstrates the feasibility of scalable, automated clinical auditing and provides a foundation for systematic quality improvement in electronic health record documentation.