Deep Sound Change: Deep and Iterative Learning, Convolutional Neural Networks, and Language Change

TL;DR

This paper introduces a novel framework using deep generative models, specifically GANs, to simulate and analyze the process of sound change in language through iterative learning across generations.

Contribution

It presents a new approach combining deep learning and iterative transmission modeling to simulate phonetic and phonological language change without relying on annotated data.

Findings

GANs can model phonetic shifts in speech data.

Iterative learning leads to gradual loss of allophonic distinctions.

Emergence of rule loss resembling phonological change.

Abstract

This paper proposes a framework for modeling sound change that combines deep learning and iterative learning. Acquisition and transmission of speech is modeled by training generations of Generative Adversarial Networks (GANs) on unannotated raw speech data. The paper argues that several properties of sound change emerge from the proposed architecture. GANs (Goodfellow et al. 2014 arXiv:1406.2661, Donahue et al. 2019 arXiv:1705.07904) are uniquely appropriate for modeling language change because the networks are trained on raw unsupervised acoustic data, contain no language-specific features and, as argued in Begu\v{s} (2020 arXiv:2006.03965), encode phonetic and phonological representations in their latent space and generate linguistically informative innovative data. The first generation of networks is trained on the relevant sequences in human speech from TIMIT. The subsequent generations are not trained on TIMIT, but on generated outputs from the previous generation and thus start learning from each other in an iterative learning task. The initial allophonic distribution is progressively being lost with each generation, likely due to pressures from the global distribution of aspiration in the training data. The networks show signs of a gradual shift in phonetic targets characteristic of a gradual phonetic sound change. At endpoints, the outputs superficially resemble a phonological change -- rule loss.