An Exploratory Study on Human-Robot Interaction using Semantics-based Situational Awareness

TL;DR

This study explores how high-level semantic understanding in robots can improve human-robot team performance by reducing workload, increasing trust, and enabling quicker decision-making in complex, high-stress scenarios like disaster response.

Contribution

We introduce a semantics-based framework that demonstrates the benefits of high-level environmental semantics in enhancing human-robot interaction and situational awareness.

Findings

Semantics reduce perceived workload of human operators.

High-level semantics increase trust in the system.

Semantics help reduce reaction time for autonomy switching.

Abstract

In this paper, we investigate the impact of high-level semantics (evaluation of the environment) on Human-Robot Teams (HRT) and Human-Robot Interaction (HRI) in the context of mobile robot deployments. Although semantics has been widely researched in AI, how high-level semantics can benefit the HRT paradigm is underexplored, often fuzzy, and intractable. We applied a semantics-based framework that could reveal different indicators of the environment (i.e. how much semantic information exists) in a mock-up disaster response mission. In such missions, semantics are crucial as the HRT should handle complex situations and respond quickly with correct decisions, where humans might have a high workload and stress. Especially when human operators need to shift their attention between robots and other tasks, they will struggle to build Situational Awareness (SA) quickly. The experiment suggests that the presented semantics: 1) alleviate the perceived workload of human operators; 2) increase the operator's trust in the SA; and 3) help to reduce the reaction time in switching the level of autonomy when needed. Additionally, we find that participants with higher trust in the system are encouraged by high-level semantics to use teleoperation mode more.