LCDS: A Logic-Controlled Discharge Summary Generation System Supporting Source Attribution and Expert Review

TL;DR

LCDS is a system that improves discharge summary generation by constraining content scope, ensuring source attribution, and supporting expert review to reduce hallucinations in LLM outputs.

Contribution

It introduces a logic-controlled framework with source mapping and attribution, enhancing reliability and reviewability of discharge summaries generated by LLMs.

Findings

Enhanced source attribution for generated summaries

Reduced hallucination issues in discharge summaries

Facilitated expert review and iterative improvement

Abstract

Despite the remarkable performance of Large Language Models (LLMs) in automated discharge summary generation, they still suffer from hallucination issues, such as generating inaccurate content or fabricating information without valid sources. In addition, electronic medical records (EMRs) typically consist of long-form data, making it challenging for LLMs to attribute the generated content to the sources. To address these challenges, we propose LCDS, a Logic-Controlled Discharge Summary generation system. LCDS constructs a source mapping table by calculating textual similarity between EMRs and discharge summaries to constrain the scope of summarized content. Moreover, LCDS incorporates a comprehensive set of logical rules, enabling it to generate more reliable silver discharge summaries tailored to different clinical fields. Furthermore, LCDS supports source attribution for generated content, allowing experts to efficiently review, provide feedback, and rectify errors. The resulting golden discharge summaries are subsequently recorded for incremental fine-tuning of LLMs. Our project and demo video are in the GitHub repository https://github.com/ycycyc02/LCDS.