Large Language Models based ASR Error Correction for Child Conversations

TL;DR

This paper investigates the effectiveness of Large Language Models in correcting errors in automatic speech recognition of children's conversational speech, highlighting their strengths and limitations in different scenarios.

Contribution

It is the first to systematically evaluate LLMs for ASR error correction specifically in child speech and conversational contexts, revealing both potential and challenges.

Findings

LLMs improve zero-shot ASR error correction in child speech.

Fine-tuned LLMs help with CTC-based ASR outputs.

Challenges remain in improving ASR with contextual info and autoregressive models.

Abstract

Automatic Speech Recognition (ASR) has recently shown remarkable progress, but accurately transcribing children's speech remains a significant challenge. Recent developments in Large Language Models (LLMs) have shown promise in improving ASR transcriptions. However, their applications in child speech including conversational scenarios are underexplored. In this study, we explore the use of LLMs in correcting ASR errors for conversational child speech. We demonstrate the promises and challenges of LLMs through experiments on two children's conversational speech datasets with both zero-shot and fine-tuned ASR outputs. We find that while LLMs are helpful in correcting zero-shot ASR outputs and fine-tuned CTC-based ASR outputs, it remains challenging for LLMs to improve ASR performance when incorporating contextual information or when using fine-tuned autoregressive ASR (e.g., Whisper) outputs.