Language Acquisition is Embodied, Interactive, Emotive: a Research Proposal

TL;DR

This research proposal emphasizes the importance of embodiment, interaction, and emotion in language learning, proposing a multimodal, grounded model to improve language acquisition in robots, inspired by human development.

Contribution

It introduces a novel semantic model integrating embodiment and emotion, and outlines a robot-dialogue system for embodied language learning based on current transformer models.

Findings

Literature review on embodiment and emotion in language learning

Proposed a grounded semantic model using transformer architectures

System design for embodied language acquisition in robots

Abstract

Humans' experience of the world is profoundly multimodal from the beginning, so why do existing state-of-the-art language models only use text as a modality to learn and represent semantic meaning? In this paper we review the literature on the role of embodiment and emotion in the interactive setting of spoken dialogue as necessary prerequisites for language learning for human children, including how words in child vocabularies are largely concrete, then shift to become more abstract as the children get older. We sketch a model of semantics that leverages current transformer-based models and a word-level grounded model, then explain the robot-dialogue system that will make use of our semantic model, the setting for the system to learn language, and existing benchmarks for evaluation.