Causal Structure Discovery for Error Diagnostics of Children's ASR

TL;DR

This paper introduces a causal discovery approach to analyze interdependent factors affecting children's ASR performance, providing insights into how physiology, cognition, and extrinsic factors influence errors and how fine-tuning mitigates these effects.

Contribution

It presents a novel causal structure discovery method for understanding complex interdependencies in children's ASR errors, extending analysis to fine-tuned models.

Findings

Causal relationships among physiological, cognitive, and extrinsic factors are identified.

Fine-tuning reduces certain factors' impact but not all.

Results are consistent across Whisper and Wav2Vec2.0 models.

Abstract

Children's automatic speech recognition (ASR) often underperforms compared to that of adults due to a confluence of interdependent factors: physiological (e.g., smaller vocal tracts), cognitive (e.g., underdeveloped pronunciation), and extrinsic (e.g., vocabulary limitations, background noise). Existing analysis methods examine the impact of these factors in isolation, neglecting interdependencies-such as age affecting ASR accuracy both directly and indirectly via pronunciation skills. In this paper, we introduce a causal structure discovery to unravel these interdependent relationships among physiology, cognition, extrinsic factors, and ASR errors. Then, we employ causal quantification to measure each factor's impact on children's ASR. We extend the analysis to fine-tuned models to identify which factors are mitigated by fine-tuning and which remain largely unaffected. Experiments on Whisper and Wav2Vec2.0 demonstrate the generalizability of our findings across different ASR systems.