IgnitionInnovators at "Discharge Me!": Chain-of-Thought Instruction Finetuning Large Language Models for Discharge Summaries

TL;DR

This paper introduces an LLM-based framework utilizing chain-of-thought prompting to improve the generation of discharge summaries, specifically targeting the 'Brief Hospital Course' and 'Discharge Instructions' sections, demonstrating enhanced reasoning and accuracy.

Contribution

The work presents a novel application of chain-of-thought prompting and instruction fine-tuning for clinical discharge summary generation, improving structural correctness and faithfulness.

Findings

Chain-of-Thought prompts improve reasoning in LLMs for clinical text generation.

Structured output prompts enhance the accuracy of discharge summaries.

Experimental results show improved structural correctness and clinical faithfulness.

Abstract

This paper presents our proposed approach to the Discharge Me! shared task, collocated with the 23th Workshop on Biomedical Natural Language Processing (BioNLP). In this work, we develop an LLM-based framework for solving the Discharge Summary Documentation (DSD) task, i.e., generating the two critical target sections `Brief Hospital Course' and `Discharge Instructions' in the discharge summary. By streamlining the recent instruction-finetuning process on LLMs, we explore several prompting strategies for optimally adapting LLMs to specific generation task of DSD. Experimental results show that providing a clear output structure, complimented by a set of comprehensive Chain-of-Thoughts (CoT) questions, effectively improves the model's reasoning capability, and thereby, enhancing the structural correctness and faithfulness of clinical information in the generated text. Source code is available at: https://github.com/antangrocket1312/Discharge_LLM