uMedSum: A Unified Framework for Advancing Medical Abstractive Summarization

TL;DR

uMedSum is a new modular framework for medical abstractive summarization that effectively balances faithfulness and informativeness, outperforming previous methods across multiple datasets and metrics.

Contribution

The paper introduces uMedSum, a novel hybrid summarization framework with a comprehensive benchmark, addressing faithfulness and informativeness in medical summarization.

Findings

uMedSum outperforms previous SOTA in quantitative metrics.

Doctors prefer uMedSum summaries 6 times more in challenging cases.

Significant 11.8% relative improvement in reference-free metrics.

Abstract

Medical abstractive summarization faces the challenge of balancing faithfulness and informativeness. Current methods often sacrifice key information for faithfulness or introduce confabulations when prioritizing informativeness. While recent advancements in techniques like in-context learning (ICL) and fine-tuning have improved medical summarization, they often overlook crucial aspects such as faithfulness and informativeness without considering advanced methods like model reasoning and self-improvement. Moreover, the field lacks a unified benchmark, hindering systematic evaluation due to varied metrics and datasets. This paper addresses these gaps by presenting a comprehensive benchmark of six advanced abstractive summarization methods across three diverse datasets using five standardized metrics. Building on these findings, we propose uMedSum, a modular hybrid summarization framework that introduces novel approaches for sequential confabulation removal followed by key missing information addition, ensuring both faithfulness and informativeness. Our work improves upon previous GPT-4-based state-of-the-art (SOTA) medical summarization methods, significantly outperforming them in both quantitative metrics and qualitative domain expert evaluations. Notably, we achieve an average relative performance improvement of 11.8% in reference-free metrics over the previous SOTA. Doctors prefer uMedSum's summaries 6 times more than previous SOTA in difficult cases where there are chances of confabulations or missing information. These results highlight uMedSum's effectiveness and generalizability across various datasets and metrics, marking a significant advancement in medical summarization.