CLINICSUM: Utilizing Language Models for Generating Clinical Summaries from Patient-Doctor Conversations

TL;DR

ClinicSum is a new framework that combines retrieval and fine-tuned language models to automatically generate accurate clinical summaries from patient-doctor conversations, improving over existing methods.

Contribution

It introduces a novel two-module architecture and a new dataset for training and evaluating clinical summarization models, advancing automated medical documentation.

Findings

Outperforms state-of-the-art PLMs in automatic metrics.

Achieves high scores in human expert assessments.

Demonstrates robustness and accuracy in clinical summarization.

Abstract

This paper presents ClinicSum, a novel framework designed to automatically generate clinical summaries from patient-doctor conversations. It utilizes a two-module architecture: a retrieval-based filtering module that extracts Subjective, Objective, Assessment, and Plan (SOAP) information from conversation transcripts, and an inference module powered by fine-tuned Pre-trained Language Models (PLMs), which leverage the extracted SOAP data to generate abstracted clinical summaries. To fine-tune the PLM, we created a training dataset of consisting 1,473 conversations-summaries pair by consolidating two publicly available datasets, FigShare and MTS-Dialog, with ground truth summaries validated by Subject Matter Experts (SMEs). ClinicSum's effectiveness is evaluated through both automatic metrics (e.g., ROUGE, BERTScore) and expert human assessments. Results show that ClinicSum outperforms state-of-the-art PLMs, demonstrating superior precision, recall, and F-1 scores in automatic evaluations and receiving high preference from SMEs in human assessment, making it a robust solution for automated clinical summarization.