Developmentally-plausible Working Memory Shapes a Critical Period for Language Acquisition

TL;DR

This paper introduces a developmental working memory mechanism into language model training, mimicking human critical periods, which improves syntactic learning efficiency and offers insights into language acquisition processes.

Contribution

It presents a novel training method that dynamically constrains working memory, aligning model development with human developmental stages to enhance language learning.

Findings

Outperforms static and unconstrained memory models in syntactic tasks

Provides evidence linking developmental working memory to critical period effects

Suggests new directions for data-efficient language model design

Abstract

Large language models possess general linguistic abilities but acquire language less efficiently than humans. This study proposes a method for integrating the developmental characteristics of working memory during the critical period, a stage when human language acquisition is particularly efficient, into the training process of language models. The proposed method introduces a mechanism that initially constrains working memory during the early stages of training and gradually relaxes this constraint in an exponential manner as learning progresses. Targeted syntactic evaluation shows that the proposed method outperforms conventional methods without memory constraints or with static memory constraints. These findings not only provide new directions for designing data-efficient language models but also offer indirect evidence supporting the role of the developmental characteristics of working memory as the underlying mechanism of the critical period in language acquisition.