Paying Attention to Function Words

TL;DR

This paper explores how the universal distinction between content and function words in natural languages can emerge through reinforcement learning in agents engaged in signaling games across varied contexts.

Contribution

It demonstrates a mechanism by which the division between content and function words can arise naturally in language evolution models.

Findings

The distinction emerges through reinforcement learning in agents.

The model applies to contexts with multiple objects and properties.

Results suggest a plausible pathway for language structure evolution.

Abstract

All natural languages exhibit a distinction between content words (like nouns and adjectives) and function words (like determiners, auxiliaries, prepositions). Yet surprisingly little has been said about the emergence of this universal architectural feature of natural languages. Why have human languages evolved to exhibit this division of labor between content and function words? How could such a distinction have emerged in the first place? This paper takes steps towards answering these questions by showing how the distinction can emerge through reinforcement learning in agents playing a signaling game across contexts which contain multiple objects that possess multiple perceptually salient gradable properties.