Unsupervised Online Grounding of Natural Language during Human-Robot Interactions

TL;DR

This paper introduces an unsupervised, online framework for grounding natural language in robot perception, enabling real-time learning of word-percept associations during human-robot interactions without prior training.

Contribution

It presents a novel cross-situational learning approach that allows online, unsupervised grounding of synonyms and phrases during interactions, outperforming existing offline methods.

Findings

Successfully grounded words in real-time during interactions

Outperformed baseline unsupervised grounding frameworks

Demonstrated robustness in online, unsupervised settings

Abstract

Allowing humans to communicate through natural language with robots requires connections between words and percepts. The process of creating these connections is called symbol grounding and has been studied for nearly three decades. Although many studies have been conducted, not many considered grounding of synonyms and the employed algorithms either work only offline or in a supervised manner. In this paper, a cross-situational learning based grounding framework is proposed that allows grounding of words and phrases through corresponding percepts without human supervision and online, i.e. it does not require any explicit training phase, but instead updates the obtained mappings for every new encountered situation. The proposed framework is evaluated through an interaction experiment between a human tutor and a robot, and compared to an existing unsupervised grounding framework. The results show that the proposed framework is able to ground words through their corresponding percepts online and in an unsupervised manner, while outperforming the baseline framework.