Towards Using Diachronic Distributed Word Representations as Models of Lexical Development

TL;DR

This paper introduces a novel diachronic distributed word representation approach to model and analyze the temporal development of children's lexical knowledge using corpus slicing and curriculum learning.

Contribution

It is the first to apply temporally sliced corpora and diachronic embeddings for modeling lexical development and transfer in children.

Findings

Semantic and syntactic knowledge acquisition trajectories visualized.

Input word frequency influences rate of word acquisition.

Lexical transfer from adults to children analyzed through representational similarity.

Abstract

Recent work has shown that distributed word representations can encode abstract information from child-directed speech. In this paper, we use diachronic distributed word representations to perform temporal modeling and analysis of lexical development in children. Unlike all previous work, we use temporally sliced corpus to learn distributed word representations of child-speech and child-directed speech under a curriculum-learning setting. In our experiments, we perform a lexical categorization task to plot the semantic and syntactic knowledge acquisition trajectories in children. Next, we perform linear mixed-effects modeling over the diachronic representational changes to study the role of input word frequencies in the rate of word acquisition in children. We also perform a fine-grained analysis of lexical knowledge transfer from adults to children using Representational Similarity Analysis. Finally, we perform a qualitative analysis of the diachronic representations from our model, which reveals the grounding and word associations in the mental lexicon of children. Our experiments demonstrate the ease of usage and effectiveness of diachronic distributed word representations in modeling lexical development.