Perception Point: Identifying Critical Learning Periods in Speech for Bilingual Networks

TL;DR

This paper investigates how deep neural visual lip-reading models develop speech perception abilities, revealing parallels with cognitive psychology's critical periods in bilingual and monolingual infants through experiments on large datasets.

Contribution

It introduces a novel analysis linking neural network development in speech perception to cognitive psychology theories of critical periods in language acquisition.

Findings

Strong correlation between cognitive theories and neural model development

Models exhibit phases similar to human speech perception development

Insights into critical periods for bilingual and monolingual speech perception

Abstract

Recent studies in speech perception have been closely linked to fields of cognitive psychology, phonology, and phonetics in linguistics. During perceptual attunement, a critical and sensitive developmental trajectory has been examined in bilingual and monolingual infants where they can best discriminate common phonemes. In this paper, we compare and identify these cognitive aspects on deep neural-based visual lip-reading models. We conduct experiments on the two most extensive public visual speech recognition datasets for English and Mandarin. Through our experimental results, we observe a strong correlation between these theories in cognitive psychology and our unique modeling. We inspect how these computational models develop similar phases in speech perception and acquisitions.