Babysit A Language Model From Scratch: Interactive Language Learning by Trials and Demonstrations

TL;DR

This paper introduces a trial-and-demonstration framework for interactive language learning in neural models, demonstrating that social-like feedback accelerates word acquisition and improves learning efficiency.

Contribution

It presents a novel interactive learning paradigm with trials and demonstrations, showing its effectiveness in enhancing language acquisition in models from scratch.

Findings

Interactive feedback accelerates word learning.

Both trials and demonstrations are crucial for efficiency.

Teacher's word choices impact learning outcomes.

Abstract

Humans are efficient language learners and inherently social creatures. Our language development is largely shaped by our social interactions, for example, the demonstration and feedback from caregivers. Contrary to human language learning, recent advancements in large language models have primarily adopted a non-interactive training paradigm, and refined pre-trained models through feedback afterward. In this work, we explore how corrective feedback from interactions influences neural language acquisition from scratch through systematically controlled experiments, assessing whether it contributes to word learning efficiency in language models. We introduce a trial-and-demonstration (TnD) learning framework that incorporates three distinct components: student trials, teacher demonstrations, and a reward conditioned on language competence at various developmental stages. Our experiments reveal that the TnD approach accelerates word acquisition for student models of equal and smaller numbers of parameters, and we highlight the significance of both trials and demonstrations. We further show that the teacher's choices of words influence students' word-specific learning efficiency, and a practice-makes-perfect effect is evident by a strong correlation between the frequency of words in trials and their respective learning curves. Our findings suggest that interactive language learning, with teacher demonstrations and active trials, can facilitate efficient word learning in language models.