MEDSAGE: Enhancing Robustness of Medical Dialogue Summarization to ASR Errors with LLM-generated Synthetic Dialogues

TL;DR

This paper introduces MEDSAGE, a novel method using Large Language Models to generate synthetic ASR errors for data augmentation, significantly improving the robustness of medical dialogue summarization systems against ASR noise.

Contribution

MEDSAGE is the first approach to leverage LLMs for generating synthetic ASR errors in medical dialogues, enhancing summarization robustness without requiring extensive audio data.

Findings

LLMs can effectively model ASR noise in medical dialogues.

Synthetic noisy data improves summarization accuracy and robustness.

The approach outperforms baseline methods in noisy ASR conditions.

Abstract

Automatic Speech Recognition (ASR) systems are pivotal in transcribing speech into text, yet the errors they introduce can significantly degrade the performance of downstream tasks like summarization. This issue is particularly pronounced in clinical dialogue summarization, a low-resource domain where supervised data for fine-tuning is scarce, necessitating the use of ASR models as black-box solutions. Employing conventional data augmentation for enhancing the noise robustness of summarization models is not feasible either due to the unavailability of sufficient medical dialogue audio recordings and corresponding ASR transcripts. To address this challenge, we propose MEDSAGE, an approach for generating synthetic samples for data augmentation using Large Language Models (LLMs). Specifically, we leverage the in-context learning capabilities of LLMs and instruct them to generate ASR-like errors based on a few available medical dialogue examples with audio recordings. Experimental results show that LLMs can effectively model ASR noise, and incorporating this noisy data into the training process significantly improves the robustness and accuracy of medical dialogue summarization systems. This approach addresses the challenges of noisy ASR outputs in critical applications, offering a robust solution to enhance the reliability of clinical dialogue summarization.