Benchmarking Training Paradigms, Dataset Composition, and Model Scaling for Child ASR in ESPnet

TL;DR

This paper evaluates training strategies, dataset effects, and model scaling for child speech recognition in ESPnet, revealing biases in SSL representations and the benefits of flat-start training, with insights into model size and open-data models.

Contribution

It provides a comprehensive benchmark comparing training paradigms, dataset compositions, and model scales specifically for child ASR, highlighting biases and optimal strategies.

Findings

SSL representations are biased toward adult speech.

Flat-start training reduces biases in child speech recognition.

Model performance improves with size up to 1B parameters, then plateaus.

Abstract

Despite advancements in ASR, child speech recognition remains challenging due to acoustic variability and limited annotated data. While fine-tuning adult ASR models on child speech is common, comparisons with flat-start training remain underexplored. We compare flat-start training across multiple datasets, SSL representations (WavLM, XEUS), and decoder architectures. Our results show that SSL representations are biased toward adult speech, with flat-start training on child speech mitigating these biases. We also analyze model scaling, finding consistent improvements up to 1B parameters, beyond which performance plateaus. Additionally, age-related ASR and speaker verification analysis highlights the limitations of proprietary models like Whisper, emphasizing the need for open-data models for reliable child speech research. All investigations are conducted using ESPnet, and our publicly available benchmark provides insights into training strategies for robust child speech processing.