Semantic-Aware Environment Perception for Mobile Human-Robot Interaction

TL;DR

This paper introduces a vision-based system for mobile robots that enhances environment perception by semantically understanding objects and humans, facilitating more natural human-robot interactions in real-world scenarios.

Contribution

The paper presents a novel semantic-aware perception system for mobile robots that operates without prior knowledge, demonstrated on a humanoid robot in real-world environments.

Findings

Effective semantic object and human registration in real-world settings

Enhanced interaction capabilities for mobile robots

System deployment on a humanoid robot validates practical applicability

Abstract

Current technological advances open up new opportunities for bringing human-machine interaction to a new level of human-centered cooperation. In this context, a key issue is the semantic understanding of the environment in order to enable mobile robots more complex interactions and a facilitated communication with humans. Prerequisites are the vision-based registration of semantic objects and humans, where the latter are further analyzed for potential interaction partners. Despite significant research achievements, the reliable and fast registration of semantic information still remains a challenging task for mobile robots in real-world scenarios. In this paper, we present a vision-based system for mobile assistive robots to enable a semantic-aware environment perception without additional a-priori knowledge. We deploy our system on a mobile humanoid robot that enables us to test our methods in real-world applications.