Spatial Concept Acquisition for a Mobile Robot that Integrates Self-Localization and Unsupervised Word Discovery from Spoken Sentences

TL;DR

This paper introduces a nonparametric Bayesian method enabling a mobile robot to learn place-related words from speech and use them for improved self-localization, demonstrated through simulations and real-world experiments.

Contribution

It presents a novel unsupervised learning approach that integrates spatial concept acquisition with self-localization, advancing robot language understanding and navigation capabilities.

Findings

Robot successfully learned place names from speech.

Acquired spatial concepts improved localization accuracy.

Method effective in both simulation and real environments.

Abstract

In this paper, we propose a novel unsupervised learning method for the lexical acquisition of words related to places visited by robots, from human continuous speech signals. We address the problem of learning novel words by a robot that has no prior knowledge of these words except for a primitive acoustic model. Further, we propose a method that allows a robot to effectively use the learned words and their meanings for self-localization tasks. The proposed method is nonparametric Bayesian spatial concept acquisition method (SpCoA) that integrates the generative model for self-localization and the unsupervised word segmentation in uttered sentences via latent variables related to the spatial concept. We implemented the proposed method SpCoA on SIGVerse, which is a simulation environment, and TurtleBot2, which is a mobile robot in a real environment. Further, we conducted experiments for evaluating the performance of SpCoA. The experimental results showed that SpCoA enabled the robot to acquire the names of places from speech sentences. They also revealed that the robot could effectively utilize the acquired spatial concepts and reduce the uncertainty in self-localization.