An iterated learning model of language change that mixes supervised and unsupervised learning

TL;DR

This paper presents a new iterated learning model for language transmission that combines supervised and unsupervised neural network training, improving computational efficiency and better mimicking child language learning processes.

Contribution

It introduces a neural network-based model using both supervised and unsupervised learning, replacing the previous reliance on obversion, to simulate language evolution more realistically.

Findings

Linear relationship between meaning-signal space and bottleneck size

Autoencoder-based model reduces computational complexity

Highlights importance of internal reflection in language learning

Abstract

The iterated learning model is an agent model which simulates the transmission of of language from generation to generation. It is used to study how the language adapts to pressures imposed by transmission. In each iteration, a language tutor exposes a na\"ive pupil to a limited training set of utterances, each pairing a random meaning with the signal that conveys it. Then the pupil becomes a tutor for a new na\"ive pupil in the next iteration. The transmission bottleneck ensures that tutors must generalize beyond the training set that they experienced. Repeated cycles of learning and generalization can result in a language that is expressive, compositional and stable. Previously, the agents in the iterated learning model mapped signals to meanings using an artificial neural network but relied on an unrealistic and computationally expensive process of obversion to map meanings to signals. Here, both maps are neural networks, trained separately through supervised learning and together through unsupervised learning in the form of an autoencoder. This avoids the computational burden entailed in obversion and introduces a mixture of supervised and unsupervised learning as observed during language learning in children. The new model demonstrates a linear relationship between the dimensionality of meaning-signal space and effective bottleneck size and suggests that internal reflection on potential utterances is important in language learning and evolution.