Human-like Linguistic Biases in Neural Speech Models: Phonetic Categorization and Phonotactic Constraints in Wav2Vec2.0

TL;DR

This study investigates how Wav2Vec2.0 neural speech models exhibit human-like biases in phonotactic constraints, revealing early-layer processing of phonological information and the influence of fine-tuning.

Contribution

It demonstrates that neural speech models encode phonotactic biases similar to humans and localizes this knowledge to early Transformer layers using controlled stimuli.

Findings

Wav2Vec2.0 shows bias towards permissible phonotactic categories.

Bias emerges in early Transformer layers.

Fine-tuning amplifies the phonotactic bias.

Abstract

What do deep neural speech models know about phonology? Existing work has examined the encoding of individual linguistic units such as phonemes in these models. Here we investigate interactions between units. Inspired by classic experiments on human speech perception, we study how Wav2Vec2 resolves phonotactic constraints. We synthesize sounds on an acoustic continuum between /l/ and /r/ and embed them in controlled contexts where only /l/, only /r/, or neither occur in English. Like humans, Wav2Vec2 models show a bias towards the phonotactically admissable category in processing such ambiguous sounds. Using simple measures to analyze model internals on the level of individual stimuli, we find that this bias emerges in early layers of the model's Transformer module. This effect is amplified by ASR finetuning but also present in fully self-supervised models. Our approach demonstrates how controlled stimulus designs can help localize specific linguistic knowledge in neural speech models.