Failing Forward: Improving Generative Error Correction for ASR with Synthetic Data and Retrieval Augmentation

TL;DR

This paper introduces DARAG, a novel method that enhances generative error correction for ASR by using synthetic data and retrieval augmentation, significantly improving performance in both in-domain and out-of-domain scenarios.

Contribution

DARAG combines synthetic data generation and retrieval-augmented correction to improve GEC for ASR, addressing generalization issues and handling unseen errors and named entities.

Findings

Achieves 8-30% relative WER reduction in in-domain scenarios.

Achieves 10-33% relative WER reduction in out-of-domain scenarios.

Outperforms baseline methods across multiple datasets and settings.

Abstract

Generative Error Correction (GEC) has emerged as a powerful post-processing method to enhance the performance of Automatic Speech Recognition (ASR) systems. However, we show that GEC models struggle to generalize beyond the specific types of errors encountered during training, limiting their ability to correct new, unseen errors at test time, particularly in out-of-domain (OOD) scenarios. This phenomenon amplifies with named entities (NEs), where, in addition to insufficient contextual information or knowledge about the NEs, novel NEs keep emerging. To address these issues, we propose DARAG (Data- and Retrieval-Augmented Generative Error Correction), a novel approach designed to improve GEC for ASR in in-domain (ID) and OOD scenarios. We augment the GEC training dataset with synthetic data generated by prompting LLMs and text-to-speech models, thereby simulating additional errors from which the model can learn. For OOD scenarios, we simulate test-time errors from new domains similarly and in an unsupervised fashion. Additionally, to better handle named entities, we introduce retrieval-augmented correction by augmenting the input with entities retrieved from a database. Our approach is simple, scalable, and both domain- and language-agnostic. We experiment on multiple datasets and settings, showing that DARAG outperforms all our baselines, achieving 8\% -- 30\% relative WER improvements in ID and 10\% -- 33\% improvements in OOD settings.