A Theory of Language Learning

TL;DR

This paper presents a Bayesian-based theory of language learning that models how children acquire language, emphasizing robustness, speed, and the ability to learn complex structures from minimal examples, aligning well with key language acquisition facts.

Contribution

It introduces a novel Bayesian framework using feature structures and scripts for language learning, capable of bootstrapping language from zero vocabulary with minimal data.

Findings

Learns complex meanings and syntax from about six examples

Robust to noise and irrelevant data

Aligns with key facts of language acquisition across multiple linguistic domains

Abstract

A theory of language learning is described, which uses Bayesian induction of feature structures (scripts) and script functions. Each word sense in a language is mentally represented by an m-script, a script function which embodies all the syntax and semantics of the word. M-scripts form a fully-lexicalised unification grammar, which can support adult language. Each word m-script can be learnt robustly from about six learning examples. The theory has been implemented as a computer model, which can bootstrap-learn a language from zero vocabulary. The Bayesian learning mechanism is (1) Capable: to learn arbitrarily complex meanings and syntactic structures; (2) Fast: learning these structures from a few examples each; (3) Robust: learning in the presence of much irrelevant noise, and (4) Self-repairing: able to acquire implicit negative evidence, using it to learn exceptions. Children learning language are clearly all of (1) - (4), whereas connectionist theories fail on (1) and (2), and symbolic theories fail on (3) and (4). The theory is in good agreement with many key facts of language acquisition, including facts which are problematic for other theories. It is compared with over 100 key cross-linguistic findings about acquisition of the lexicon, phrase structure, morphology, complementation and control, auxiliaries, verb argument structures, gaps and movement - in nearly all cases giving unforced agreement without extra assumptions.