Unified Understanding of Environment, Task, and Human for Human-Robot Interaction in Real-World Environments

TL;DR

This paper presents an integrated system enabling service robots to understand, adapt, and communicate effectively during human-robot interactions in dynamic real-world environments, demonstrated through a restaurant scenario.

Contribution

The study introduces a novel indoor dynamic map, task understanding, and response generation system tailored for real-world HRI tasks, especially in dynamic settings like restaurants.

Findings

Achieved 90% accuracy in serving and communication tasks in simulated environments.

Received an average rating of 4.2 out of 5 in user questionnaires.

Demonstrated effective HRI for waiter duties in a dynamic environment.

Abstract

To facilitate human--robot interaction (HRI) tasks in real-world scenarios, service robots must adapt to dynamic environments and understand the required tasks while effectively communicating with humans. To accomplish HRI in practice, we propose a novel indoor dynamic map, task understanding system, and response generation system. The indoor dynamic map optimizes robot behavior by managing an occupancy grid map and dynamic information, such as furniture and humans, in separate layers. The task understanding system targets tasks that require multiple actions, such as serving ordered items. Task representations that predefine the flow of necessary actions are applied to achieve highly accurate understanding. The response generation system is executed in parallel with task understanding to facilitate smooth HRI by informing humans of the subsequent actions of the robot. In this study, we focused on waiter duties in a restaurant setting as a representative application of HRI in a dynamic environment. We developed an HRI system that could perform tasks such as serving food and cleaning up while communicating with customers. In experiments conducted in a simulated restaurant environment, the proposed HRI system successfully communicated with customers and served ordered food with 90\% accuracy. In a questionnaire administered after the experiment, the HRI system of the robot received 4.2 points out of 5. These outcomes indicated the effectiveness of the proposed method and HRI system in executing waiter tasks in real-world environments.