BabAR: from phoneme recognition to developmental measures of young children's speech production

TL;DR

This paper introduces BabAR, a multilingual phoneme recognition system trained on a large child speech corpus, enabling scalable analysis of early speech development and correlating with developmental milestones.

Contribution

We created TinyVox, a large multilingual child speech corpus, and developed BabAR, a cross-linguistic phoneme recognition system that improves with multilingual pretraining and contextual fine-tuning.

Findings

Pretraining on multilingual data significantly improves recognition accuracy.

Contextual audio during fine-tuning enhances performance.

BabAR's measures align with established developmental benchmarks.

Abstract

Studying early speech development at scale requires automatic tools, yet automatic phoneme recognition, especially for young children, remains largely unsolved. Building on decades of data collection, we curate TinyVox, a corpus of more than half a million phonetically transcribed child vocalizations in English, French, Portuguese, German, and Spanish. We use TinyVox to train BabAR, a cross-linguistic phoneme recognition system for child speech. We find that pretraining the system on multilingual child-centered daylong recordings substantially outperforms alternatives, and that providing 20 seconds of surrounding audio context during fine-tuning further improves performance. Error analyses show that substitutions predominantly fall within the same broad phonetic categories, suggesting suitability for coarse-grained developmental analyses. We validate BabAR by showing that its automatic measures of speech maturity align with developmental estimates from the literature.