A Technique for Isolating Lexically-Independent Phonetic Dependencies in Generative CNNs

TL;DR

This paper investigates how generative CNNs trained on raw audio can generalize phonotactic rules independently of lexical content, introducing a novel probing technique that reveals convolutional layers' capacity for phonetic dependency generalization.

Contribution

It presents a new method for probing lexically-independent phonetic generalizations in CNNs, especially under a narrow fully-connected layer bottleneck, demonstrating dynamic phonetic dependency generalization.

Findings

Convolutional layers can generalize phonetic dependencies beyond lexical constraints.

A novel probing technique works only with a narrow FC bottleneck.

CNNs trained on raw audio exhibit lexically-invariant phonotactic generalizations.

Abstract

The ability of deep neural networks (DNNs) to represent phonotactic generalizations derived from lexical learning remains an open question. This study (1) investigates the lexically-invariant generalization capacity of generative convolutional neural networks (CNNs) trained on raw audio waveforms of lexical items and (2) explores the consequences of shrinking the fully-connected layer (FC) bottleneck from 1024 channels to 8 before training. Ultimately, a novel technique for probing a model's lexically-independent generalizations is proposed that works only under the narrow FC bottleneck: generating audio outputs by bypassing the FC and inputting randomized feature maps into the convolutional block. These outputs are equally biased by a phonotactic restriction in training as are outputs generated with the FC. This result shows that the convolutional layers can dynamically generalize phonetic dependencies beyond lexically-constrained configurations learned by the FC.