Modelling Compositionality and Structure Dependence in Natural Language

TL;DR

This paper formalizes the concepts of compositionality and structure dependence in natural language, grounding linguistic theory in set theory, and demonstrates a cognitively plausible model using word embeddings and role-filler binding.

Contribution

It introduces a formal set-theoretic framework for linguistic properties and develops a relation learning model that aligns with cognitive constraints.

Findings

The formal model captures key linguistic constraints.

A custom dataset demonstrates the model's ability to learn relations.

The model shows structure mapping with a symbolic-connectionist architecture.

Abstract

Human beings possess the most sophisticated computational machinery in the known universe. We can understand language of rich descriptive power, and communicate in the same environment with astonishing clarity. Two of the many contributors to the interest in natural language - the properties of Compositionality and Structure Dependence, are well documented, and offer a vast space to ask interesting modelling questions. The first step to begin answering these questions is to ground verbal theory in formal terms. Drawing on linguistics and set theory, a formalisation of these ideas is presented in the first half of this thesis. We see how cognitive systems that process language need to have certain functional constraints, viz. time based, incremental operations that rely on a structurally defined domain. The observations that result from analysing this formal setup are examined as part of a modelling exercise. Using the advances of word embedding techniques, a model of relational learning is simulated with a custom dataset to demonstrate how a time based role-filler binding mechanism satisfies some of the constraints described in the first section. The model's ability to map structure, along with its symbolic-connectionist architecture makes for a cognitively plausible implementation. The formalisation and simulation are together an attempt to recognise the constraints imposed by linguistic theory, and explore the opportunities presented by a cognitive model of relation learning to realise these constraints.