Real-Time Multimodal Signal Processing for HRI in RoboCup: Understanding a Human Referee

TL;DR

This paper presents a real-time multimodal signal processing system for RoboCup, enabling robots to interpret referee gestures and whistles accurately, thereby improving autonomous human-robot interaction in dynamic, competitive environments.

Contribution

It introduces a two-stage gesture recognition pipeline and CCNN-based whistle detection tailored for real-time HRI in RoboCup, with minimal reliance on network connectivity.

Findings

Enhanced real-time gesture recognition accuracy

Effective whistle detection with CCNNs

Improved HRI performance in RoboCup scenarios

Abstract

Advancing human-robot communication is crucial for autonomous systems operating in dynamic environments, where accurate real-time interpretation of human signals is essential. RoboCup provides a compelling scenario for testing these capabilities, requiring robots to understand referee gestures and whistle with minimal network reliance. Using the NAO robot platform, this study implements a two-stage pipeline for gesture recognition through keypoint extraction and classification, alongside continuous convolutional neural networks (CCNNs) for efficient whistle detection. The proposed approach enhances real-time human-robot interaction in a competitive setting like RoboCup, offering some tools to advance the development of autonomous systems capable of cooperating with humans.