Identity-Based Patterns in Deep Convolutional Networks: Generative Adversarial Phonology and Reduplication

TL;DR

This paper demonstrates that deep convolutional neural networks can learn and manipulate identity-based patterns like reduplication in speech, revealing insights into neural network interpretability and representation of linguistic patterns.

Contribution

It introduces a novel technique to test CNNs' understanding of reduplication and shows how latent space manipulation can control pattern generation in speech.

Findings

CNNs learn to represent identity-based patterns in latent space.

Manipulating latent variables can produce reduplicated speech forms.

The network generalizes the pattern to unobserved data.

Abstract

This paper models unsupervised learning of an identity-based pattern (or copying) in speech called reduplication from raw continuous data with deep convolutional neural networks. We use the ciwGAN architecture Begu\v{s} (2021a; arXiv:2006.02951) in which learning of meaningful representations in speech emerges from a requirement that the CNNs generate informative data. We propose a technique to wug-test CNNs trained on speech and, based on four generative tests, argue that the network learns to represent an identity-based pattern in its latent space. By manipulating only two categorical variables in the latent space, we can actively turn an unreduplicated form into a reduplicated form with no other substantial changes to the output in the majority of cases. We also argue that the network extends the identity-based pattern to unobserved data. Exploration of how meaningful representations of identity-based patterns emerge in CNNs and how the latent space variables outside of the training range correlate with identity-based patterns in the output has general implications for neural network interpretability.